Applications: The Now and the Future.
Heyo folks. We've had a couple discussions about applications — [1] and [2] — over the last year. The introduction of BN Mentorship and its rising popularity have brought into question the utility of Applications as a whole.

Based on these discussions, the current state of the community, the current state (and size!) of the group, and based on some data we collected, we would like to use this post as to bring some changes onto the table, as well as explaining our current state and what changes we could realistically make.

We believe a discussion, before pushing any changes, is necessary — your opinion as an up-to-be applicant or current BN who has survived the gauntlet matters!

The Data
Just so we have some data, to preface our current situation. Here are the graphs with the applications passed/failed both per mode and across all modes within the last 365 days (June 20th, 2024 — May 23rd, 2025).


The following charts have not taken the following into consideration:

1. Behaviour denials & failed background checks: Users solely denied for behaviour, or if the Account support team had probable cause to deny them. These fail conditions would occur regardless of the system used, so they're not relevant to us now. These amounted to 10 in total: 8 for osu!, 1 for osu!taiko, and the last one for osu!mania.
2. Mentorship additions: Mentorship will be discussed later. Only osu!, osu!taiko, and osu!mania added users through mentorship, totaling 40, 1, and 12 users respectively.
3. Re-Additions in Good Terms: Users leaving in Good Terms do not have to go through the same application as other users do. Those who rejoin within a year of leaving in Good Terms, unless they have misbehaved, will get in 100% of the times. Across all four modes (O!TCM), there have been a total of 22, 10, 8, and 13 re-additions following this process. Some of them being the same users multiple times.

On top of this, we've run The Numbers™. Here's a table with the amount of BNs per mode, number of NAT members per mode, and the ratio between BN:NAT — AKA how many Beatmap Nominators per NAT member currently exist.



The data shown above is very important to understand our point of view, and why some of the ideas previously suggested, at this point in time, are currently not sustainable or not possible to implement at this point in time.

So... what's the plan?
We've drawn the following conclusions from the data above, as well as other available data such as the amount of Beatmaps being ranked or the workload we, as a team, have to tackle:

1. The Ratio in osu! is currently not sustainable. While this is actively being fixed by the most recent wave of BN Evaluators, with the team bolstering some of their numbers, adding more Beatmap Nominators would prove to be unsustainable with the current Evaluation model.
2. Mentorship has had a noticeable impact in The Numbers™. This is not to say the additions were done haphazardly or unjustified. However, we've sort of experienced our own "BN Baby Boom" with the introduction of Mentorship.
3. The overall influx of maps being ranked is quite positive. The total sets being Ranked on a per month basis is steadily increasing, with no signs of stopping. The increase in group size has had a tangible impact on the amount of sets being ranked. This carries its own set of problems, as the Ranking Queue is constantly saturated with sets despite the buffer's increase.

To sum it up, we do believe there is an issue with Applications. While this is not to say all Fails were unwarranted, the Pass/Fail ratio itself as opposed to Mentorship's is something worth noting. However, changes must be done very carefully given the other observations brought up.

There are active efforts going into Evaluations and how we conduct them, in the hopes the process can be streamlined for both Evaluators and Beatmap Nominators. Or, in short, avoid evaluating users who've proven to be working just fine without any issues!

As for Mentorship, while there are no plans to stop it any time soon, we're in the process of streamlining and standardizing the addition process. This means that mentees will most likely require to pass some sort of final test rather than being able to pass by the Mentors' sole decree. This is pending to be discussed! So unless stated otherwise, Mentorship will work as-is.

Proposed changes: Now & Tomorrow
That said, we believe there are some changes that Applications in particular could boast of. While it may not be the best times to implement changes that may make access easier, we believe making Applications more accessible — not to be confused with "Easier" — is going to be a net positive for future applicants.

Here's the list of changes we wish to bring to the table, along with those we believe are worth exploring at a later date when the group size is a bit more stable:

1. Creation of an Evaluation/Application Rubric. This Rubric would be an aid to both applicants and Evaluators, who'd have a common frame of reference to both understand what an applicant needs to show in their application and what an Evaluator needs to look for in said application. While the idea is to use this as an aid rather than the deciding factor of an addition, this should hopefully make the learning process easier, as well as making it more digestible for the end-user when obtaining feedback from Evaluators with the exact points to improve on. This rubric would be public!
2. Increase Probation length to 2-3 months. This would allow us to lower the barrier of entry; allowing a similar flow as that introduced by Trial BNs back when. While at first this change may not appear to be substantial, in practice it would allow Evaluators to be more trustworthy of applicants. By giving applicants more time during Probation, fringe cases that would've otherwise been denied in the past would have their chance to shine and improve during Probation. The sequence diagram below shows one potential use case, with a more hand-holdy approach in the hopes to ease in new nominators, and to allow newcomers to iterate their learning in a more AGILE manner.
   
   
3. (On Hold) Rework the Application's questions & required amount of maps. Before implementing this, we'd like to give the two ideas above a solid shot. The idea with this would be to rework the application's questions to be more "atomic", allowing users to submit their mods piece-by-piece, asking and evaluating their rationale even more than we do already, etc... In short, the goal is to make the application more accessible by being able to DIY your full mods, rather than search for The Perfect Map for an application, which admittedly has caused some headaches.
4. (On Hold) Taking into account a user's previous experiences. Applications are, by design, isolated from everything else a user has to offer. In turn, this makes it possible for someone with a lot of Mapping/Modding experience (not necessarily geared towards being a BN necessarily) and yet fail an Application. This, admittedly, sucks. While we have no concrete suggestions, finding a way to incorporate a user's previous experience throughout the community in some capacity would hopefully make it easier for users who've proven themselves elsewhere to give 'em a shot at the whole Nominating shebang.

We're open to discuss these. Keeping in mind the situation and data shown above, we'd like to know your thoughts on the team's best course of action!

Conclusion
All in all, we believe the current amount of Beatmap Nominators is sufficiently large. And while Applications nor Mentorship will not be closing any time soon, we have to tread carefully with more additions until we can stabilize both the BN:NAT ratios and optimize our internal workloads, particularly Evaluations.

heavily agree with simplifying/automating the NAT evaluation work of BN, if someone has a low DQ ratio, then dont evaluate them unless they had been involved in VETO, save urself some time, this has to be done to increase the NAT to BN ratio

Adding more BN doesn't simply increase more maps being ranked, but rather increase the variety of song/maps being ranked, which is very important to satisfy players of all types

the Ranking Queue is constantly saturated with sets despite the buffer's increase. -> i dont get why this is a problem? this is just growth and its good

agree with everything else

Increasing the probation period sounds good if you’re also lowering the standards when accepting the new applicants.

Mentorships are good but ppl are starting to rely too much on this to get into the bng.

I like this. I believe this is a step in the right direction.

I think it would be ideal for the BN evaluator cycles to keep a similar rhythm as it is now until the BN/NAT ratio stabilizes itself, perhaps aiming for a BN/NAT ratio of around 10 or 8 ideally.

I find the probation length increase something *really* positive and glad that some parts of my proposal were considered.

Not gonna lie, I'm a little intrigued with this new evaluation sequence for probation BNs, it would essentially give some extra work and activity requirement when compared to the current system, but curious to see how well that system would prove itself overtime in combination with the other changes.

Also, the "On Hold" topics would be *huge* game-changers and I'm really excited to see those being implemented.

why not add more nat

Ngl the proposed changes sound amazing for BN Applicants and Probationary BNs. But if the NAT were to be more lenient on BNs applicants (while the probation would be "harder" given that more applications/longer time = more chances to fuck up), won't it make the workload for the NAT significantly more important?

The ratio of BNs to NAT in standard is already huge, the NAT already evaluate ton of full BNs, so I'm slightly concerned about an even higher workload for evaluations - from an outsider's perspective.

These are definitely changes that were long overdue. Great work!

Although, I have a question: when candidates enter extended probation, they’re presumably scrutinized for the quality of their nominations—but wouldn’t that also reflect poorly on the BNs who nominated those maps with them? I mean, even if that feedback doesn’t affect the BNs’ actual performance (since there are no map-quality evaluations), it still makes it clear that the NAT considers their nominations questionable, to say the least. I say this because I can already see the conflict between the lack of internal standards and the expectations set for new candidates—especially when they nominate controversial maps.

I believe the issue you reached on conclusion is only an issue because of how unoptimized the eval work is done.

Other than that, great to see changes for the app system, glad we don't want to gatekeep BNs by joining only by mentorship and you guys acknowledge that reality.

These changes are amazing. I think one of the big things about being a bn is the mental it takes to keep up with the workflow. This is something that is very difficult for applications to encompass to any extent. Huge +1 to increasing probation duration. At least in regards to catch, I think it would allow for a better reflection of ones abilities (imo).

I have mixed feelings about these potential changes, so I'll write my thoughts for each one separately.

RandomeLoL wrote:

Creation of an Evaluation/Application Rubric

Very positive change. Currently applicants kinda have to hit their heads against the wall blindly because it's hard to find out what exactly NAT expect from your mods. Even with feedback, there are usually no clear directions, and a lot of applicants keep repeating the same mistakes. Surely it's not always because they learn that slowly.

RandomeLoL wrote:

Increase Probation length to 2-3 months

Honestly, don't think this is beneficial. If NAT want to watch a new BN more closely for a longer period of time, it can be done behind the scenes without stressing them out with frequent official evals. Restrictions that probo BNs have are quite annoying too.

Longer probo period can only be helpful if there's some guidance during that period, like a lighter version of mentorship maybe. Especially if the apps are treated with more leniency, new BNs will really need that guidance, not just extended probation.

RandomeLoL wrote:

Rework the Application's questions

Might help a lot if those questions really make applicants assemble a BN mod bit by bit. However it needs to be balanced carefully not to overcomplicate the app itself, otherwise it will be a return of BN tests.

RandomeLoL wrote:

Taking into account a user's previous experiences

Imo previous experience shouldn't affect anything except a cooldown in case of a fail. If applicant's experience is really making it easier for them to learn BN stuff, they'll either pass right away or will be ready to reapply soon. But it's not guaranteed, so why gamble and give a pass to someone who clearly failed an app.

Usaha wrote:

why not add more nat

Yeah, BN/NAT ratio being problematic was mentioned so many times but the most obvious solution wasn't.

I do agree with some of the proposed changes for one reason, ease of access of information to the new public as well making the NAT and evaluators lives easier when it comes to doing the job. That said, I feel like there are some things that need further explanation for me to grasp and I think would be curiosities brought up by others as well:

RandomeLoL wrote:

1. The Ratio in osu! is currently not sustainable. While this is actively being fixed by the most recent wave of BN Evaluators, with the team bolstering some of their numbers, adding more Beatmap Nominators would prove to be unsustainable with the current Evaluation model.

1. What's currently going wrong with the evaluation process that makes the ratio not sustainable?

I understand the the influx of members has increased significantly with the introduction on BN mentorship which could cause a bottleneck when it comes to the processing of evaluations BUT... I would like to have a better insight in the problematic of the evaluations from staff side as well since that wasn't expanded on and I think that discussions would be more fruitful if we know the core issue of evaluations themselves that are not sustainable so we can properly provide solutions to this.

For what is worth, this is something that I think needs to be expanded on for more clarity and transparency on the current Evaluation model so we the participants can give an even better feedback.

RandomeLoL wrote:

1. Creation of an Evaluation/Application Rubric. This Rubric would be an aid to both applicants and Evaluators, who'd have a common frame of reference to both understand what an applicant needs to show in their application and what an Evaluator needs to look for in said application. While the idea is to use this as an aid rather than the deciding factor of an addition, this should hopefully make the learning process easier, as well as making it more digestible for the end-user when obtaining feedback from Evaluators with the exact points to improve on. This rubric would be public!

1. How would the rubric contents be determined? Is it arbitrary content or will there be proper discussions across all game modes for a draft creation?

Mentioning this part since there are mostly just NATs doing evaluations at the moment and I do believe the the discussions mentioned at the start of the thread are valid when it comes to applying feedback. Sometimes feedback can be subjective or objective or just plain neutral but when it comes to getting an eval, it could mean all 3 of those are going to be combined and as discussion [2], the BN gets nothing at all.

As an ex-NAT that was involved in the evaluation process before, I felt that that it was very streamlined but with everything in this game and in life, it must have evolved by then to be a process quite difficult for the influx of members so how would this learning curve of the new Evaluation model affect current and future evaluations?

RandomeLoL wrote:

1. Increase Probation length to 2-3 months. This would allow us to lower the barrier of entry; allowing a similar flow as that introduced by Trial BNs back when. While at first this change may not appear to be substantial, in practice it would allow Evaluators to be more trustworthy of applicants. By giving applicants more time during Probation, fringe cases that would've otherwise been denied in the past would have their chance to shine and improve during Probation. The sequence diagram below shows one potential use case, with a more hand-holdy approach in the hopes to ease in new nominators, and to allow newcomers to iterate their learning in a more AGILE manner.

1. How will bad actors factor in with the increased probation time and with the implied fact that standards are going to be toned down to give the user a proper chance to try out? Will the NAT also have more leniency to deal with such cases?

I think this is something more of a touchy subject since regardless of, we allow users to become Beatmap Nominators with the (I wouldn't say hope, but for a lack of a better word at the moment) hope or trust that they will act in good faith towards the community and abide by the CoC, Beatmap Nominator Rules and the Expectations for Beatmap Nominators.

Since the overall goal for this is to also make applications a center of influx of members into the BNG, with the new proposed changes, how would bad actors come into play is one of my concerns. Ideally, the NAT would have more leniency to kick people in those cases but what about people that lay on the low and create trouble coming right out their probation period?

I understand that behavior and previous account status are taken into account but it's a worry I can't shake off just yet without having some reassurance.


Good work on the proposal and drafting up this discussion.

EDIT: I do believe the addition of more NATs could aide the workflow issue as well.

Lot of ground to cover, so bear with me. Do want to note that these are mostly my own views on the matter based on all discussions I've been involved in. Other team members might have a slightly different take on the matter. Especially on a per-mode basis.

Drum-Hitnormal wrote:

the Ranking Queue is constantly saturated with sets despite the buffer's increase. -> i dont get why this is a problem? this is just growth and its good

Growth is good up to a certain point. I guess the answer to this question lies in the reason why the queue was originally implemented in the first place. Its current purpose is two-fold: avoids saturating one day with a lot of maps in favour of spreading them through a longer period of time, plus it allows maps to have more time during Qualified and encourage QA.

WALL-E wrote:

...won't it make the workload for the NAT significantly more important?

Neto wrote:

I believe the issue you reached on conclusion is only an issue because of how unoptimized the eval work is done.

It may have been missed amidst all of the other text, but Evaluations and other internal workflows are actively being discussed. Ideally, if we can manage to make these processes more efficient, we'd be able to procure more time for this approach to applications. So yes, this is an issue. But not an overlooked one.

melleganol wrote:

when candidates enter extended probation, they’re presumably scrutinized for the quality of their nominations—but wouldn’t that also reflect poorly on the BNs who nominated those maps with them?

The issues in regards to "standards" and "quality" are far outside the scope of this discussion. I don't really understand the point you're trying to make though most likely. This is not a matter of nominating "controversial" maps or being more heavy-handed during probation. The point is to have a fairer assessment on an applicant.

Usaha wrote:

why not add more nat

Kurisu Makise wrote:

Yeah, BN/NAT ratio being problematic was mentioned so many times but the most obvious solution wasn't.

Antalf wrote:

I do believe the addition of more NATs could aide the workflow issue as well.

This is actively being worked by the mode who's currently suffering the most. The recent influx of BN Evaluator rounds and promotions should already be an indicator that this is being worked on.

Granted, again, this is not as easy as simply adding more people in. Adding people to the team is a solution — and it's actively being worked on. But it's not a long term solution to a team's overall sustainability.

Kurisu Makise wrote:

If NAT want to watch a new BN more closely for a longer period of time, it can be done behind the scenes without stressing them out with frequent official evals.

Then we run into a problem where BNs are not getting a "fair" assessment, and we're not being transparent to them. The point of the mini-evals is precisely to offer that guidance to newcomers. More frequent feedback, and faster learning iterations.

Kurisu Makise wrote:

If applicant's experience is really making it easier for them to learn BN stuff, they'll either pass right away or will be ready to reapply soon.

From our experience, this hasn't been the case. Users with all the experience and modding backbone in the world can still fail an application. This is what that point was trying to make. Applications feel disconnected from your day to day modding workflows.

Antalf wrote:

What's currently going wrong with the evaluation process that makes the ratio not sustainable?

I'd prefer if an osu! NAT were to give a proper answer to this, as they're the ones most affected by it. Modes like osu!mania are overall more than fine on this regard, for example. The problem doesn't need to be expanded more than just stating there's too many BNs for so little NAT members to evaluate them on time.

Antalf wrote:

How would the rubric contents be determined? Is it arbitrary content or will there be proper discussions across all game modes for a draft creation?

These will just be made out of the information already found throughout the wiki and with the criteria we've been following so far. Pages like Becoming a Beatmap Nominator or the guide found in the BN Site itself already list all of the expectations we have. We're not reinventing the wheel nor changing the evaluation criteria. I don't get what you mean by "getting nothing at all", but it is worth emphasizing that the rubric is not a replacement for written feedback nor other aspects not codified in a piece of paper. Consider this an aid for both applicants (which will dispose of a handy "checklist" or summary) and evaluators (who will have an easier task checking for specific parts in an applicant's app).

Antalf wrote:

How will bad actors factor in with the increased probation time and with the implied fact that standards are going to be toned down to give the user a proper chance to try out? Will the NAT also have more leniency to deal with such cases?

This is irrelevant. Bad actors will be dealt with no matter what system is used. This leniency is only applied to the modding itself. This does not mean we're going to be any less lax on behavioural concerns, following the BN Rules, Code of Conduct, or expectations alike. Let it be during the application, Probation, or as full-fledged BNs.

Good changes. Seems like it's long been overdue. Though I'd like to ask the possibility of the mentorship program being absorbed into Probation instead.

Basically, instead of having a separate track, I'd much rather that the mentorship aims to drag the BNs up to speed after the BN application, where the ones "competently able to mod" are filtered out immediately.

As of currently, as a BN Mentorship graduate, the whole system feels very similar to Mentorship and it feels redundant if the applications after changes get an influx of people passing.

What are the rubric contents:

It's a more written out version of the application expectations that have been on the wiki for ages.


Why not just add more NAT:

Doing so carelessly will only cause different issues to appear or expand in the future by throwing people not equipped for it at it, leading to issues in teamwork, accuracy, consistency.

Just as we can't expand BN endlessly, we can't necessarily do that for NAT either - the bigger a group is, the harder teamwork and coordination can get, you have to consider how exactly that will work first, it's not a simple maths game, unfortunately. Especially one for a group that is expected to be somewhat able to see eye to eye as an entire group instead of working as solo individuals.

But the current standard ratio is just simply, really, bad, and not manageable at all in the other direction. This really can cause problems if the workflow hasn't changed much. We have fewer standard NAT than many previous years and twice as many BNs.

We did not have mock evals or bn evaluators for a while due to various reasons, such as figuring out a general direction, and what would we be doing, what would we expect of additions, etc...

But if it wasn't evident already, the poor size is already why we have made the move to have bn evaluator cycles basically back to back in standard, and will continue to do so, whether that's just for letting people see what it is, practice that perspective, or to add new people. For as long as it is an issue this will continue to be the case.


E:
And I don't mean this to say to be dismissive of anything, because above all we want to help people succeed and pass their apps and nominate maps and many more things - just, it's important to be aware of where things stand to make actually doable plans. Some require more steps than others to work first.

It makes sense that NAT are very careful expanding their group. But on the other hand, BNG is expanding disproportionately and these upcoming changes can only accelerate the process. Shouldn't BN recruitment be handled with the same care?

I'm gonna say something that can be considered harsh or even hypocritical coming from a BN. I apologize in advance if it offends anyone and I swear that I only have mapping community's well-being in mind.

We need more normal modders, not BNs. Idk what's modding situation in other modes but in mania it's already at the point where mappers struggle to find non-BN mods. And without non-BN mods you can only rank your map if you're very good at mapping or if you find a hero BN who will solo carry your map to the state where 2nd BN can just do a normal check.

Lowering the standard and accepting more probo BNs will further decrease amount of normal modders and raise amount of BNs who are not able to solo carry any map. Is that really what we want?

That being said, I'm all for making apps more accessible and providing all the necessary information to applicants. First of all, it's just fair and a right thing to do. Second, if more people learn and reach the current BN standard rather than lowered one, it can't be bad.

@kurisu makise

i'm not nat, but here's my views:

in osu!std gamemode, the BN recruitment has been hit-or-miss. while i'd say that the recruitment is handled with enough care, it's ineffective, and that's what I believe this post/change is trying to address.

depending on how probation is implemented (I would assume similarly to the BN mentorship where modding is heavily required), the new wave of probationary BNs will have a necessity to mod properly, and the expectation is passed down to the applicant to be able to mod sets competently, like a BN, before being accepted into the BNG. it may be idealistic, but unironically, the new system may foster new, and competent normal modders willing to become BN.

i can't say for mania, but considering that osu!std is the largest gamemode, I believe that the std! mapping community can be a case study for the other gamemodes that will inevitably grow larger and likely suffer the same problems that the std! gamemode currently has.

Thank you for mentioning this flawed system, honestly, nowadays app system is pretty unreasonable when normal modder applies BN application (not just limiting mania mode.)

Even I entered from BN mentorship and only experienced 2 app attempts, current app pass standards are extremely high and applicants can't think where the criteria is and overthink how NATs would satisfy our modding. Sometimes, leaning to neutral-fail applicants' modding skill is going worse and got bad score compared with previous one and gives up to do BN. This is NOT a good situation for everyone to see.

From applicant's view, current BN app and BN mentorship program's positions are swapped, BN app is dealt with as the side road and BN mentorship is the main road to go BN. All NATs say BN mentorship is not a "shortcut" to go BN; however, as we see app accept ratio, their mind won't be changed unless the app system is reworked.

If lowering their criteria and giving more chances with longer time to oversee them, I can agree with this.

ON HOLD stuffs: application questions are worse than before IMO, they are forced to find spread map with NHI and not to accept, ... etc, this can't cover all situations and is ridiculously hard to find suitable map, especially if the applicant is aiming to do anti-meta main (example one is non-4K in mania.)
Also previous experiences; I've seen this kind of accept reason in 2021~2022 era, and this logic also applies to former BNs who aren't getting "good term" period recently, I think this is not a bad idea to check applicants. Neutral but leaning to agree this

my 2c:
feel very skeptical about this, 2-3 months probation sounds unnecessary excessive for something as nominating maps, like it's the same duration as internship for some actual, real life jobs where you have more responsibilities than looking for wrong offset; I'd even argue that if you need this much time on probation you probably aren't ready to be a bn yet

another issue I see with the new probation is that it may be incredibly frustrating and stressful for both parties to have multiple monthly evaluations and have some bns fail after literally months of probation, which, again, is rather too long given we're talking about nominating maps in std

the only issues I have with the current apps are: a)looking for specific maps (reworking the app questions may address this) b)the disconnect between apps and bn work that happens since on apps you're actually expected to be able to mod maps to some extent, while as a bn your main kpi is the ratio of dqs/unrankables you missed and their severity to your overall activity, which can be solved by either evaluating bns' modding/maps they nominate, or reworking apps in such a way that focuses on applicant's ability to check unrankables rather than their modding

Is there increased demand in ranked section to warrant larger BNG roster? The player population is stagnant and slowly declining, which is contrary to BN headcount.

timemon wrote:

Is there increased demand in ranked section to warrant larger BNG roster? The player population is stagnant and slowly declining, which is contrary to BN headcount.

I believe the playerbase issue is unrelated to this topic. However, if you get the map submission data, it increased hugely. Nowadays there are more mappers, more complete mapsets and an insane demand for ranking maps compared to the past.

This seems like a nice thing, applications definitely need some discussion, this seems like it's steering into the right direction.

I would not extend the probation period for more than 2 months if at all though.

as a mapper and someone who is willing to become a BN one day, i really like this proposal quite a lot & would like to see this in the near future. however, as stated by multiple ppl above, there is one thing i do agree with, and it's the fact that there should be more BNs added to the NAT. i feel like the current amount of NATs on the team isn't enough when you compare it to the huge amount of Full BNs, and should increase, since mentoring & keeping track of what all the BNs do can be a rough challenge to keep up with with a small team.

if anything, its best to maybe consider balancing both BN and NAT teams for standard since the ratio between BNs & NATs are really huge atm.

This sounds quite good as proposal and discussion, anyway i do agree about the Nekro doubt. 2-3 months of probationary Is quite long even It would be very helpful to test the new applicant, but i think you're going to dig a hole under your feet if you have to evaluate all probs every month, counting full bn's evals and new applicants' evals. So i would keep in mind about this change cause the large ratio present on STD would become very bloated.

Rest i think it's fine and, if i don't understand something, i think i'll find It

- Rubric -
Yes, great idea.

- Longer Probation -
No, allowing more time for evaluations does not solve the problem of having too many evaluations to do in the first place. Building the app rubric (potentially even a separate evaluation rubric for NATs' reference) for this purpose could streamline the evaluation process and make it more objective. I think you should sit on this idea and see if it's still necessary after some other changes and time have passed.

> "If NAT want to watch a new BN more closely for a longer period of time, it can be done behind the scenes without stressing them out with frequent official evals." @Kurisu

This, too. More checks means more NAT workload. And at the end of the day, BN is unpaid volunteer work. This and the proposed additional checks is just doing too much imo.

... A suggestion to reduce the quantity of evaluation numbers would be to not do it at all, except in the case of reports/resets. Maybe volunteer GMT can assist or smth.

- Rework App Questions -
No, just give them the rubric and allow applicants more slots to meet all the requirements. Like they MUST send 3 maps, but may include up to like 5 in order to touch on every point that you want to see.

- Previous Experience -
No, being good at mapping does not mean you are good at teaching or spotting unrankables. In fact, if it were possible, I think it'd be ideal if the modding quality of applications could be reviewed anonymously, then do the behaviour check after. I don't think that's practical, but it conveys the spirit of my stance hopefully.

...

> "We need more normal modders, not BNs." also @Kurisu
Maybe some dev-side QOL changes could help encourage modding? Like making it easier to link pictures and files. Maybe give active modders some form of clout. Exactly how a system like that would be set up is a topic for another thread. But I quote this because it suggests some people who are going for BN shouldn't be pursuing that. Initially that sounds morbid, but when the big-picture problem is having too many BNs, I think incentivizing a different role is worth exploring.

@NAT

1. It seems you don't want to further upset the ratio by accepting too many more BNs for the moment. Making the application process clearer is good to do, but it's a separate topic. It won't fix this. Yes, do that anyways.
2. Noffy says getting more NATs is a complicated, time-consuming process to do correctly. OK sounds good, do the vetting correctly. But no plans have been mentioned to reduce the size of the BNG, either, so the ratio cannot be fixed within a short time. Unless you can trust them to run completely autonomously - undermining NAT's purpose - streamlining the evaluation processes seems to be the only solution.
3. You don't want to close applications. Why not?

Everyone jokes about the quality of std ranked maps these days. How seriously people, particularly BNs and NATs, feel about this I don't know. But this thread seems to be structured around finding a way to make the same processes - which people are already unsatisfied with - work, just on a bigger scale. Is that the case, or are BNs and NATs generally satisfied with the system? If not, with such large-scale restructuring on the table, this could be an opportunity to reestablish quality standards more people are happy with.
...Kinda separate, do NAT members actually want subjectivity influence back?

I don't know if and to what extent this has been discussed but the main problem with the current NAT workload is simply the fact that evaluations are way too frequent, especially for BNs who perform well and have years of experience.
I really don't think it's necessary to evaluate every single BN every 3 months, even in an actual job setting people aren't evaluated so frequently, it's usually just once or maybe twice a year. Reducing the amount of evals would not only make NAT work easier but also relieve BNs of the constant stress/anxiety of getting a negative judgement.

In practice, the frequency could scale based on how long someone has been a BN and how well they've been doing in the past. For example, the first evaluation after passing probation could be done 3 months later (just like now), but after that it would be done 6 months later, and then every 12 months. If someone hasn't been doing well, it would go back to the standard 3 months and so on.

To answer the question "Wouldn't underperforming BNs go unnoticed for too long?", there are 2 things to consider:

1. The NAT can still monitor nomination activity and amount of DQs regularly (this is already automated to my knowledge, at least for the activity part). If a BN isn't performing well, the NAT can, at any time, talk to them to understand what the issue might be and discuss if/how it can be solved. If necessary, they can give the BN a warning and in very severe cases, the BN may even be removed, but I think punishment should be a last resort and not the first course of action (like it kind of is now).
2. This issue can be avoided or at least alleviated by focusing more on prevention rather than reaction, which means that BNs should be helped and guided more during probation so they can learn how to avoid certain mistakes or deal with certain issues, as well as getting better in areas they're less experienced in. Additionally, implementing a working QA system could prevent bad nominations from getting ranked instead of finding out about them a few weeks/months later when the map is already ranked and the mistakes can't be fixed anymore, which is one of the major flaws of the current approach. With this sort of preventive action, even if evals occur less often, there wouldn't be the risk of an underperforming BN pushing maps to ranked that aren't ready or have unrankables for a long period of time.

Regarding the data, it's very nice to have official numbers, but I feel like the graphs alone are a bit difficult to interpret, would it be possible to add it in terms of percentages as well (such as x% of applications passed for example)?

The rubric sounds like a good idea if done properly, but it's hard to judge based on this description so hopefully there will be some examples soon.

I actually think the 2 points which are marked as "on hold" are among the core issues that have plagued BN applications for years so I believe it would be crucial to further discuss and work on them as soon as possible. I'm not sure how exactly the ideas for changing the application questions would look like and whether these changes would ve beneficial, but there are definitely several possibilities of improving this system, some of which have been suggested in other threads before.

As one of the few people who recently got into the bng the "raw" way, through the application and without previous bn experience i absolutely approve the idea of increasing the amount of maps in the application, as it was my main struggle point - to find that exact set of maps which would provide me with exact content i want in my bn app. It's very frustrating to be bottlenecked not by your modding ability, or even the language barrier, but by what basically is a side-quest consisting of scrolling through the hundreds of pending and graved maps. Increasing the amount of maps in the application also should give evaluators more context on what applicant is willing or not willing to push, thus indirectly aiding the next point you propose - all the mapping and modding experience will have much more opportunity to show itself. This will undoubtedly increase the workload of NAT and evaluators but with the drastic increase in numbers it should be fine

And also this would address the things nekro said - with more maps in the app you will have significantly less load on each individual mod, just like one or two major problems + unrankable checks, bringing app mods closer to what bns actually do

One more potential downside i can currently see is that people might get confused with what do they need so many slots for and don't utilize them effectively, but this is one of the things that the rubric shall fix i guess?

P.S - this also could improve the current landscape of modding community in general, as there are loads of very capable modders right now in search of their bn app maps who simply ignore sets they could've modded because these sets don't fit what they are searching for. More slots for the maps in bn app -> bigger frame of what could be accepted + more modded maps in general, potentially helping some mappers who might be in real need of that help

RandomeLoL wrote:

Antalf wrote:

How would the rubric contents be determined? Is it arbitrary content or will there be proper discussions across all game modes for a draft creation?

These will just be made out of the information already found throughout the wiki and with the criteria we've been following so far. Pages like Becoming a Beatmap Nominator or the guide found in the BN Site itself already list all of the expectations we have. We're not reinventing the wheel nor changing the evaluation criteria. I don't get what you mean by "getting nothing at all", but it is worth emphasizing that the rubric is not a replacement for written feedback nor other aspects not codified in a piece of paper. Consider this an aid for both applicants (which will dispose of a handy "checklist" or summary) and evaluators (who will have an easier task checking for specific parts in an applicant's app).

Antalf wrote:

How will bad actors factor in with the increased probation time and with the implied fact that standards are going to be toned down to give the user a proper chance to try out? Will the NAT also have more leniency to deal with such cases?

This is irrelevant. Bad actors will be dealt with no matter what system is used. This leniency is only applied to the modding itself. This does not mean we're going to be any less lax on behavioural concerns, following the BN Rules, Code of Conduct, or expectations alike. Let it be during the application, Probation, or as full-fledged BNs.

As for the "gets nothing at all", I am referring to the one discussion labeled [2] in the original post which is from melleganol which they point out (after I read all of it and both of them) that the amount of feedback that can vary so much since it's so subjective, the applicant wont gain anything positive or negative from seeing their feedback; which is something that I wholeheartedly agree with.

I don't think it's valid calling something "irrelevant" when it came from a place of genuine concern and doubt that I brought up after reading this discussion. This should've been more like "it's not a concern" rather than being called "irrelevant".

Antalf wrote:

I don't think it's valid calling something "irrelevant" when it came from a place of genuine concern and doubt that I brought up after reading this discussion. This should've been more like "it's not a concern" rather than being called "irrelevant".

Just a quick response to this as my point was not to belittle the argument, but rather explain how this concern is not relevant under any application model or rework. This is not to say the concern is invalid, but that it's not relevant in the current context.

Some people mentioned that a 3-month evaluation might be too long. I’d say it definitely could be, but it also might not. It really depends on how the evaluation is structured. If NAT can keep the tasks clear and manageable without adding too much extra workload, then I think it’ll work out just fine.

The solution could be to still filter for applicants with good potential and behaviour, especially the ones that needs polishing. That said, a 2-month period might be more ideal, similar to mentorship. To increase flexibility, we could give this extended evaluation option only to BNs who show strong potential but aren’t quite there yet, lowering the entry barier. Those who already solid from the start can still follow the usual 1-month evaluation path to full BN.

Sorry if I'm missing the details and context on how the application goes currently tho. Correct me if I did please

hi just stopping by to say - while I do see the concerns that 2-3 month probation would be quite long, even if it isn't exactly that system that gets put in place, I do think we're desperately in need of some sort of "in between" leniency to let people try bn. it's something that's been coming up a lot lately. whether it's still an extended probation but reduced to 2 months max, or a separate approach following in the footsteps of trial bn, I think it's an idea we shouldn't give up on yet. just want to quickly voice my support for that specifically. that's not to say I don't support the other stuff, it looks good too, just haven't had time to read through all of it yet

I mostly support these, I think these are a good step in the right direction. Though, one thing:

RandomeLoL wrote:

(On Hold) Taking into account a user's previous experiences. Applications are, by design, isolated from everything else a user has to offer. In turn, this makes it possible for someone with a lot of Mapping/Modding experience (not necessarily geared towards being a BN necessarily) and yet fail an Application. This, admittedly, sucks. While we have no concrete suggestions, finding a way to incorporate a user's previous experience throughout the community in some capacity would hopefully make it easier for users who've proven themselves elsewhere to give 'em a shot at the whole Nominating shebang.

I think this should be considered 100%, but..being a good mapper or having mapping experience doesn't make you a good modder, nor does being a good modder make you a good mapper. I would even say just being a good modder doesn't mean you would be a good BN.
But if you guys consider a user's previous experience, I think modding experience is a good start, but I just don't see how mapping experience would relate to being a good BN because they aren't the same thing, only slightly correlating at most.

As someone who has applied (and failed) 8 applications at this point, and also not been accepted into the official mentorship despite applying every iteration since it began, I think these changes look good, especially points towards the rubric, increase in probation length (and thus lowering of expectation of the actual application). The three mini evals would be an awesome addition as well, as the current application system takes a LOT of time and effort in finding and modding maps that aren't of interest of the applicant, which is one of my main problems with the application.

Increasing the numbers of NAT seems to be really needed and I am glad that that is being addressed actively.

The point for taking into account user's previous experiences I am unsure about, in an application it would be difficult to find experience outside of previously failed applications (which would benefit me so I would appreciate that being added, but it feels like a moot point that doesn't reflect the skill of the actual applicant). I would think a better way to use previous experience to bring someone into BNG is in the mentorship program, in that mentors should choose (or be assigned?) applicants with more failed applications/kudosu/something like that to then choose the people with the most experience/dedication towards modding and becoming a BN to bring them into mentorship for (what should be) a more straightforward and easier time into the BNG, rather than a game of connections or RNG to get into mentorship.

I do agree that the current group is sufficiently large, and the qualified section is often overrun by maps that is a lot of work for QA, however I wanted to ask/make note that increasing the size of NAT would directly correlate to this, as NAT are the ones responsible for QA to my knowledge, so I do believe this to also be a moot point since increasing NAT numbers is something actively being addressed.

I would love to see graphs for applications failed/passed that do not include mentorship members or previous BNs, for to my knowledge it has been a VERY long time since the previous application pass for someone outside of that select group (correct me if I'm wrong).

The main problem with bnapps currently is that it is widely accepted that "mentorship is the only way into BNG", which the fact that mentorship works well is a great thing, however because mentorship cannot accept everyone in every cycle, it creates an unfair environment to entering BNG, and either a change to make the bnapp more lenient, mentorship more open/fair, or some mixture of those would be the best way to remedy these widely accepted thoughts, which I do believe to be negative overall.

PS. wanted to also say that I agree with everything Sanch and Kataryn said as well from my perspective, as of now it feels like the application is too strict and it is difficult to know what to do as an applicant, and it feels like whether I pass or not is not my modding skill or knowledge, but instead by my map choice, which shouldn't be what the app is testing

Sorry if I'm necroposting as it's been 3 months since the last post, but I had some ideas I wanted to present regarding the NAT/BN ratio.

Since the issue is that the ratio is unsustainable, the immediate solution is to increase NAT, right? Noffy stated: "Just as we can't expand BN endlessly, we can't necessarily do that for NAT either - the bigger a group is, the harder teamwork and coordination can get, you have to consider how exactly that will work first, it's not a simple maths game, unfortunately. Especially one for a group that is expected to be somewhat able to see eye to eye as an entire group instead of working as solo individuals." Since we can't simply add more members to NAT and BN, I suggest some of these things could be implemented to prevent the ratio from getting worse:

1. Adding a rule limiting the ratio of BNs to NATs.

The rule could be made like so: " The ratio of BNs to NATs for each mode should not exceed 10. This is to prevent members of the NAT from being overworked/overwhelmed."

I thought this rule might make sense since if you compare BN to other teams such as GMT, BSC, etc., BN is the only team that is always open for applications, as opposed to the other teams mentioned that add members less frequently and more selectively, without even making use of applications most of the time. Naturally, this will cause BN to grow much more, especially on top of mentorship existing.

2. Decrease frequency of mentorship cycles/number of mentees per cycle + change NAT activity requirement.

Since mentorship seems to the biggest contributor to an increase in BNs, what if we decrease the number of mentees and frequency of cycles? As much as it would be nice to do the opposite, it would cause the ratio to become too high. Although, a counterargument could be that more BNs will automatically bring more NATs, but the number of BNs that would become NATs might not be high enough to keep the ratio down. I think something like an incentive or a lower activity requirement for NATs might help to make BNs more willing to join and make it easier for existing NATs.

Right now, NATs must nominate 2 maps per month and BNs must nominate 6 maps per 90 days. If we look at the ratio between nominations and time, they are both 2:30. The only difference is a BN could nominate all 6 maps in 1 month and be fine for the remaining 2, but an NAT can't, which actually gives BNs more leniency regarding nominations. Something that could ease NAT work, even if just a little bit, is switch the activity requirement between NATs and BNs, so that NATs have more leniency for nominations. Another measure that could be done on top of that is to lower NATs activity requirement from 2 maps to 1 map per month or even removing the requirement entirely. I don't think it would be an issue since NATs are usually a smaller group, so if they nominate less or no maps at all, it shouldn't be significant. Plus, a lot of BNs go beyond the activity requirement anyway, so that also contributes.

3. Divide NATs and BNs into divisions to prevent excessive growth and inability to function.

Since Noffy mentioned simply adding NATs isn't an option, what if we divide the group into two (three or more might be needed in the future depending on BN/NAT growth)? Let's take standard for example. We have 12 NATS and 119 BNs, including probationary BNS. We can round up to 120 for a ratio of 10, although ratio isn't the main issue here (I'm just using our current numbers as an example). If the number of NATs become too big for the group to effectively function, instead of something like 12:120, we could split it into two divisions of 6:60 and 6:60. This way, each division can function as its own group (almost as if they are their own modes?) so that NAT numbers become manageable within the group. Both BNs and NATs could be randomly assigned to each division and NATs in division 1 evaluate BNs that are also in division 1 (same for division 2).

---

So, these were my ideas about the BN/NAT ratio, and I hope they weren't too extreme or moronic. If you have any thoughts about this, feel free to share.

The Cosmic Chef wrote:

Sorry if I'm necroposting as it's been 3 months since the last post, but I had some ideas I wanted to present regarding the NAT/BN ratio.

Since the issue is that the ratio is unsustainable, the immediate solution is to increase NAT, right? Noffy stated: "Just as we can't expand BN endlessly, we can't necessarily do that for NAT either - the bigger a group is, the harder teamwork and coordination can get, you have to consider how exactly that will work first, it's not a simple maths game, unfortunately. Especially one for a group that is expected to be somewhat able to see eye to eye as an entire group instead of working as solo individuals." Since we can't simply add more members to NAT and BN, I suggest some of these things could be implemented to prevent the ratio from getting worse:

1. Adding a rule limiting the ratio of BNs to NATs.

The rule could be made like so: " The ratio of BNs to NATs for each mode should not exceed 10. This is to prevent members of the NAT from being overworked/overwhelmed."

I thought this rule might make sense since if you compare BN to other teams such as GMT, BSC, etc., BN is the only team that is always open for applications, as opposed to the other teams mentioned that add members less frequently and more selectively, without even making use of applications most of the time. Naturally, this will cause BN to grow much more, especially on top of mentorship existing.

2. Decrease frequency of mentorship cycles/number of mentees per cycle + change NAT activity requirement.

Since mentorship seems to the biggest contributor to an increase in BNs, what if we decrease the number of mentees and frequency of cycles? As much as it would be nice to do the opposite, it would cause the ratio to become too high. Although, a counterargument could be that more BNs will automatically bring more NATs, but the number of BNs that would become NATs might not be high enough to keep the ratio down. I think something like an incentive or a lower activity requirement for NATs might help to make BNs more willing to join and make it easier for existing NATs.

Right now, NATs must nominate 2 maps per month and BNs must nominate 6 maps per 90 days. If we look at the ratio between nominations and time, they are both 2:30. The only difference is a BN could nominate all 6 maps in 1 month and be fine for the remaining 2, but an NAT can't, which actually gives BNs more leniency regarding nominations. Something that could ease NAT work, even if just a little bit, is switch the activity requirement between NATs and BNs, so that NATs have more leniency for nominations. Another measure that could be done on top of that is to lower NATs activity requirement from 2 maps to 1 map per month or even removing the requirement entirely. I don't think it would be an issue since NATs are usually a smaller group, so if they nominate less or no maps at all, it shouldn't be significant. Plus, a lot of BNs go beyond the activity requirement anyway, so that also contributes.

3. Divide NATs and BNs into divisions to prevent excessive growth and inability to function.

Since Noffy mentioned simply adding NATs isn't an option, what if we divide the group into two (three or more might be needed in the future depending on BN/NAT growth)? Let's take standard for example. We have 12 NATS and 119 BNs, including probationary BNS. We can round up to 120 for a ratio of 10, although ratio isn't the main issue here (I'm just using our current numbers as an example). If the number of NATs become too big for the group to effectively function, instead of something like 12:120, we could split it into two divisions of 6:60 and 6:60. This way, each division can function as its own group (almost as if they are their own modes?) so that NAT numbers become manageable within the group. Both BNs and NATs could be randomly assigned to each division and NATs in division 1 evaluate BNs that are also in division 1 (same for division 2).

---

So, these were my ideas about the BN/NAT ratio, and I hope they weren't too extreme or moronic. If you have any thoughts about this, feel free to share.

Regarding 1: How would this rule be enforced when the ratio is exceeded? By promoting BNs, removing them, or by other means? Honestly I don't see how having such a rule solves the problem.

Regarding 2: The last BN mentorship cycle was already skipped for standard and the cycles are done less frequently than the normal mentorship program so this is already the case somewhat. Lowering the number of mentees is possible but I'm not sure if that's beneficial if the demand is high.
It's actually not true that NAT members have to nominate 2 maps per month, there is no nomination requirement for them, so this point is moot.

Regarding 3: I really don't think it makes a difference or helps to have people split into 2 groups. The total number would still be the same, and while a smaller subgroup might seem easier to manage, it would only cause further divide in the community which is already heavily devided. It also seems needlessly complicated from an organisational point of view (it would require more user groups, changes to the BN website etc.).

Serizawa Haruki wrote:

The Cosmic Chef wrote:

Sorry if I'm necroposting as it's been 3 months since the last post, but I had some ideas I wanted to present regarding the NAT/BN ratio.

Since the issue is that the ratio is unsustainable, the immediate solution is to increase NAT, right? Noffy stated: "Just as we can't expand BN endlessly, we can't necessarily do that for NAT either - the bigger a group is, the harder teamwork and coordination can get, you have to consider how exactly that will work first, it's not a simple maths game, unfortunately. Especially one for a group that is expected to be somewhat able to see eye to eye as an entire group instead of working as solo individuals." Since we can't simply add more members to NAT and BN, I suggest some of these things could be implemented to prevent the ratio from getting worse:

1. Adding a rule limiting the ratio of BNs to NATs.

The rule could be made like so: " The ratio of BNs to NATs for each mode should not exceed 10. This is to prevent members of the NAT from being overworked/overwhelmed."

I thought this rule might make sense since if you compare BN to other teams such as GMT, BSC, etc., BN is the only team that is always open for applications, as opposed to the other teams mentioned that add members less frequently and more selectively, without even making use of applications most of the time. Naturally, this will cause BN to grow much more, especially on top of mentorship existing.

2. Decrease frequency of mentorship cycles/number of mentees per cycle + change NAT activity requirement.

Since mentorship seems to the biggest contributor to an increase in BNs, what if we decrease the number of mentees and frequency of cycles? As much as it would be nice to do the opposite, it would cause the ratio to become too high. Although, a counterargument could be that more BNs will automatically bring more NATs, but the number of BNs that would become NATs might not be high enough to keep the ratio down. I think something like an incentive or a lower activity requirement for NATs might help to make BNs more willing to join and make it easier for existing NATs.

Right now, NATs must nominate 2 maps per month and BNs must nominate 6 maps per 90 days. If we look at the ratio between nominations and time, they are both 2:30. The only difference is a BN could nominate all 6 maps in 1 month and be fine for the remaining 2, but an NAT can't, which actually gives BNs more leniency regarding nominations. Something that could ease NAT work, even if just a little bit, is switch the activity requirement between NATs and BNs, so that NATs have more leniency for nominations. Another measure that could be done on top of that is to lower NATs activity requirement from 2 maps to 1 map per month or even removing the requirement entirely. I don't think it would be an issue since NATs are usually a smaller group, so if they nominate less or no maps at all, it shouldn't be significant. Plus, a lot of BNs go beyond the activity requirement anyway, so that also contributes.

3. Divide NATs and BNs into divisions to prevent excessive growth and inability to function.

Since Noffy mentioned simply adding NATs isn't an option, what if we divide the group into two (three or more might be needed in the future depending on BN/NAT growth)? Let's take standard for example. We have 12 NATS and 119 BNs, including probationary BNS. We can round up to 120 for a ratio of 10, although ratio isn't the main issue here (I'm just using our current numbers as an example). If the number of NATs become too big for the group to effectively function, instead of something like 12:120, we could split it into two divisions of 6:60 and 6:60. This way, each division can function as its own group (almost as if they are their own modes?) so that NAT numbers become manageable within the group. Both BNs and NATs could be randomly assigned to each division and NATs in division 1 evaluate BNs that are also in division 1 (same for division 2).

---

So, these were my ideas about the BN/NAT ratio, and I hope they weren't too extreme or moronic. If you have any thoughts about this, feel free to share.

Regarding 1: How would this rule be enforced when the ratio is exceeded? By promoting BNs, removing them, or by other means? Honestly I don't see how having such a rule solves the problem.

Regarding 2: The last BN mentorship cycle was already skipped for standard and the cycles are done less frequently than the normal mentorship program so this is already the case somewhat. Lowering the number of mentees is possible but I'm not sure if that's beneficial if the demand is high.
It's actually not true that NAT members have to nominate 2 maps per month, there is no nomination requirement for them, so this point is moot.

Regarding 3: I really don't think it makes a difference or helps to have people split into 2 groups. The total number would still be the same, and while a smaller subgroup might seem easier to manage, it would only cause further divide in the community which is already heavily devided. It also seems needlessly complicated from an organisational point of view (it would require more user groups, changes to the BN website etc.).

To enforce 1, I was thinking once the ratio of 10 is hit, BN applications would close so new members can't be added. Once the ratio dips down enough, applications could re-open. Although, returning BNs that can auto-rejoin might be something to consider.

For 2, stuctural NATs don't have a requirement, but the evaluation NATs do as seen here:

Nomination: Nominating at least 2 beatmaps per month. This helps evaluators keep up to date with the mapping/modding community when evaluating current and aspiring Beatmap Nominators.

As for 3, I get what you mean. I honestly think standard is just a really specific situation since we have more BNs/NATs than the other modes combined. My suggestion was mainly considering the possibility that evaluations wouldn't change to allow the mode to support a larger ratio and work more efficiently, but if that could happen, it's definitely preferable.