Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. Beatmaps
  3. Mapping Discussion
  4. Beatmap Management

[assigned] Mapping Ecosystem Changes - BN Application Process

posted
Total Posts
99
Hivie
osu!taiko Paragon
902 posts
Joined March 2019
Topic Starter
Hivie

Mapping Ecosystem Changes


The following changes serve to make contributing more easily accessible to the community, which consequently improves upon and simplifies all current systems in place for the mapping and modding community.

Mapping Ecosystem Changes - BN Application Process
Mapping Ecosystem Changes - BN Ecosystem
Mapping Ecosystem Changes - Vetoes
Mapping Ecosystem Changes - NAT Additions

Reworking BN applications


BN applications will undergo a facelift to lessen the focus on individual mods provided in the application, and will instead put more weight onto the candidate's decision-making skills and judgment.

Users will be asked to submit 3 distinct beatmaps that they have modded, and answer certain questions based on whether they would nominate said maps or not.

Therefore, the application format will be as follows:

Beatmap 1
Beatmap 2
Beatmap 3

Note: submitting oszs is not mandatory, but is recommended.

Removing the RC test


The BN test has proven to be an unnecessary overhead that's usually outdated, often serving no meaningful purpose other than being a glorified CAPTCHA by being more indicative of reading comprehension than BN abilities.

It also includes niche RC questions aren't worth being a barrier to entry.

Increasing transparency on a BN application's status


Current status of a BN application as it's progressing will be directly reported to the applicant in the form of a visible progress tracker.

Therefore, the applicant will be able to see whether their application is still in the individual stage, group stage, had its consensus set, and so on.

Reworking BN application feedbacks


Feedbacks have proven to be one of the most exhausting tasks that also caused a ton of delays and overdue evaluations.
Lots of applications also get stuck at this stage, so cutting it out significantly reduces the time between a user applying and receiving the consensus.

Instead, we will directly display the evaluator's comments directly to the applicant (while making the comment authors anonymous), which have been deemed to be detailed enough to provide meaningful feedback to the applicant.

Do note that the list of who participated in an application's evaluation will still be visible.

Platform for communication between NAT and applicant post-application


Evaluators and applicants alike voiced concerns about how it can be tiring to set up group DMs and have long discussions about disagreeing with an evaluation consensus.
So, we're providing a platform of communication directly on an applicant's evaluation results page! This will give a direct and dedicated space to formally discuss disagreements and ask questions regarding an application, without having to reach out to an NAT member privately.

Do note that applicants will still know who participated in their evaluation, therefore knowing who they're talking with.

These are not final changes, but are proposals. We highly appreciate your input whether you're supporting these, want to discuss potential issues, or provide alternative solutions/approaches!
Last edited by Hivie , edited 17 times in total.
maxie
119 posts
Joined May 2015
maxie
I assume you guys will do this but you should probably say what each of the different stages mean in an eval for the BN app transparency thing
Last edited by maxie , edited 1 time in total.
wafer
Beatmap Nominator
775 posts
Joined December 2016
wafer
I like this proposal a lot, but im kind of concerned about the execution of evaluating judgement abilities of applicants

Does this imply the standard for modding quality decreases in exchange for people that may just know how to appease the system (provide decent responses)?

How does this affect non native english speakers?



I think there should still be some baseline checking of modding quality - unless there is and I have misread .
melleganol
Beatmap Nominator
606 posts
Joined March 2020
melleganol
this is GOOD, gj all of you!
maxie
119 posts
Joined May 2015
maxie
Also, while I do not disagree that the current RC test is redundant, I think it could be useful to rework it to include relevant information. For example, I think a test so that possible BNs are aware of certain technical things would be good, specifically regarding metadata and audio quality. Maybe just have the scores not be impactful or something, but idk I just feel like the RC test could be reworked to be actually useful.
fayew
Beatmap Nominator
209 posts
Joined June 2018
fayew
i'll go over each section with my thoughts

Reworking BN applications
i'd also suggest adding difficulty ranges to showcase that possible BNs may work with any sorts of maps, being high-end (8*+) or something more casual (enh sets/typical 5-6* sets), this would also guarantee more quality overall imo. the question "how did your mod improve the map" might be a little trivial and hard to answer, since this is a quite subjective topic and not really good overall. kind of against including it. other than that this change is good

Removing the RC test
i think removing rc test should've been done way earlier, especially it comes in clutch for taiko/catch/mania tests where you're potentially able to get a question regarding skinnable std elements (that you really shouldn't know) or the other solution would be creating different tests for gamemodes that contain questions mostly regarding relative gamemodes (but i support the removal more tbh)

Increasing transparency on a BN application's status
how is this any different from what is currently done? i do not think this change is necessary, it just adds bullet point list on top of your evaluation - which might be pleasant to eyes but effectively speaking it's near the same as a collective feedback that is more gathered into one single piece of text and more polished. very against this

Platform for communication between NAT and applicant post-application
a good change, nice to see this being thought of
Okoratu
Nomination Assessment Team / Elite Nominator
4,913 posts
Joined May 2012
Okoratu
i think one of the axing reasons for test was cuz people doing tests in their freetime for a video game is kinda over the top on the serious scale dunno

re: fayew: the transparency part is so people dont like throw application into the box and receive an update like 2 to 3 weeks later, but instead have more consistent updates as to where in the process their application is, it's just more transparency for that
Last edited by Okoratu , edited 1 time in total.
Hivie
osu!taiko Paragon
902 posts
Joined March 2019
Topic Starter
Hivie

fayew wrote:

how is this any different from what is currently done?
right now, you apply, your eval goes into a black box for 2~3 weeks, then it ships. with the proposed approach, you'll know at which stage exactly your application is in real-time, so you'll know if it's still in individual stage, if it moved to groups, if the consensus is set and so on.

i really don't see a reason to be against this, it's a direct transparency increase which is always appreciated.
fayew
Beatmap Nominator
209 posts
Joined June 2018
fayew

Okoratu wrote:

the transparency part is so people dont like throw application into the box and receive an update like 2 to 3 weeks later, but instead have more consistent updates as to where in the process their application is, it's just more transparency for that

Hivie wrote:

it's a direct transparency increase which is always appreciated.
i see, thank you both
enneya
Nomination Assessment Team
374 posts
Joined October 2017
enneya

wafer wrote:

I think there should still be some baseline checking of modding quality - unless there is and I have misread .
it will still be there (especially for sets they won't nominate): this is why we opted to not completely remove the mention of modding in the application form. the wording can still adjust the wording to be a bit clearer about this if you have any propositions
Ozato Fumika
Beatmap Nominator
3,379 posts
Joined July 2013
Ozato Fumika
I like the simplified yet more engaging progress very much but just a few things I want to say.

Would this mean the future BN applicants require mapper's response and should care about their response?
as you guys are asking for how their mod improved the map as well as the updated version of osz for comparison.
For example, if a mapper decides to not respond or reject every suggestion, would this "mod" now be considered unusable for the application?

Also, rather than removing the RC test entirely, I think it would be better to give it a rework as an actually reasonable RC test instead of the previous English trick test.
wafer
Beatmap Nominator
775 posts
Joined December 2016
wafer
@enneya ok cool

ok yeah u have my full support then
Resona
Beatmap Nominator
236 posts
Joined August 2013
Resona
really dont see the point in an rc test cuz if i can ctrl+f it while taking the test i can ctrl+f it while looking at a map to nominate

agree with sampling maps that the applicant would/wouldn't nominate, and imo that's something that could be expanded on further. like listing idk 2-3 more maps that you may not mod on thread, but just give tldr of what you might say + final judgement (nom or not)
FuJu
Elite Nominator II
175 posts
Joined September 2017
FuJu

Ozato Fumika wrote:

Would this mean the future BN applicants require mapper's response and should care about their response?
as you guys are asking for how their mod improved the map as well as the updated version of osz for comparison.
For example, if a mapper decides to not respond or reject every suggestion, would this "mod" now be considered unusable for the application?
'how their mod improved the map' is supposed to be a subjective assessment so its not dependant on the mappers response. Therefore, even if the host rejects all the mods, we will still check them for validity like we always have, so it wouldn't be unusable per se.

Ozato Fumika wrote:

Also, rather than removing the RC test entirely, I think it would be better to give it a rework as an actually reasonable RC test instead of the previous English trick test.
We removed the test entirely for now because it was pretty outdated and seen as unfitting for the reason that Oko mentioned. However, since we are using the Beatmap Managment forum now, the test can still be repurposed depending on new proposals.
fvrex
Beatmap Nominator
324 posts
Joined March 2018
fvrex
I really like this proposal, I just have a question

with “Reworking BN application feedbacks”, does this mean that the box for application feedback won’t be there, and only NAT comments are visible?

if it is, I think it’s a bit redundant to remove it as it’s a really good way to synthesise the comments into a tldr, so I’d suggest both could be visible
Hivie
osu!taiko Paragon
902 posts
Joined March 2019
Topic Starter
Hivie

fvrex wrote:

with “Reworking BN application feedbacks”, does this mean that the box for application feedback won’t be there, and only NAT comments are visible?
Box will stay in case NATs have any additional notes to mention, but most of the important feedback will be displayed directly in form of evaluator comments.
Ryax
Beatmap Nominator
172 posts
Joined September 2016
Ryax
Seems like a good change overall to me, the RC test definitely had a lot of unnecessary coverage on it.

For Beatmap 1 in the application I could see some occasional issues cropping up with the 0 nomination requirement. I wouldn't want an applicant to end up being disqualified because a BN nominated the map, since that's something out of their control. In the same respect, I don't think it's fair to the mapper or BNs on the set to have to wait for the applicant's results before continuing to ranked.

I understand that this is meant to avoid having applicants pick maps that have already been approved by BNs as a safety net, but I'm a little worried about it.
Decku
Beatmap Nominator
196 posts
Joined November 2018
Decku
I do agree this proposal is a slay 🔥🔥


fvrex wrote:

I really like this proposal, I just have a question

with “Reworking BN application feedbacks”, does this mean that the box for application feedback won’t be there, and only NAT comments are visible?

if it is, I think it’s a bit redundant to remove it as it’s a really good way to synthesise the comments into a tldr, so I’d suggest both could be visible
For this by the way, basically application feedback wont be included in the evaluation process. HOWEVER, if this is the case, then the feedback should rather be applied into the actual feedback NAT evaluators are giving them during the evaluations!!! Giving more detailed feedback should be more prominent honestly.

Ryax wrote:

For Beatmap 1 in the application I could see some occasional issues cropping up with the 0 nomination requirement.

I can also see an issue with this. It does seem sort of unfair on that part if you did all the means necessary for the BN, where they hadn't checked it.
Last edited by Decku , edited 4 times in total.
Nao Tomori
Nomination Assessment Team
3,089 posts
Joined December 2014
Nao Tomori

Ryax wrote:

Seems like a good change overall to me, the RC test definitely had a lot of unnecessary coverage on it.

For Beatmap 1 in the application I could see some occasional issues cropping up with the 0 nomination requirement. I wouldn't want an applicant to end up being disqualified because a BN nominated the map, since that's something out of their control. In the same respect, I don't think it's fair to the mapper or BNs on the set to have to wait for the applicant's results before continuing to ranked.

I understand that this is meant to avoid having applicants pick maps that have already been approved by BNs as a safety net, but I'm a little worried about it.
This is a fair concern. However, the point of this is to attempt to determine the applicant's quality standards and ability to evaluate maps in a vacuum. That objective is significantly impaired if the map has already been nominated. The requirement would only extend to maps not bubbled at the time of submission, not throughout the life of the application. As such, I view this as a necessary tradeoff to better accomplish the goals of the system.
Noffy
Nomination Assessment Team
1,659 posts
Joined April 2012
Noffy

Decku wrote:

For this by the way, basically application feedback wont be included in the evaluation process. HOWEVER, if this is the case, then the feedback should rather be applied into the actual feedback NAT evaluators are giving them during the evaluations!!! Giving more detailed feedback should be more prominent honestly.
It is already the case that feedback is primarily organizing information from individual evaluations. We would of course make sure they are formatted to better communicate to the applicant rather than just each other. I believe the message box to reach out should also help in case any individual points are not clear enough right away
show more
Please sign in to reply.

New reply

0
1 / 99