forum

An Idea For a New Ranking System

posted
Total Posts
12
Topic Starter
Osuology
I realize that this is a hugely different system from the current, so it may be too much change. In that case, I may make this into a sort of third-party ranking system. I also have a discord server I made for this, but I'm not sure if I'm allowed to post the invite as that might be advertising. PM me for the link.

My idea is what I'm calling the VRF osu! system! What is this all about?
The VRF stands for: Verified, Ranked, Featured. These represent the three tiers of maps in the system. Let's go through all of those three and describe them:

Verified
Verified: If a map is verified, that means it has been evaluated by the Verification Team or VT. The VT judges each map submitted to them by the following criteria:
-The map must have 75% mapped, or no parts of the song are left unmapped that deserve to be mapped, as judged by the VT.
-The map must be properly timed, with proper bpm and offset for every timing marker.
-The map must be literally possible to complete (it must not autofail or be passable by auto).
-The map must not be inappropriate, and this will be decided by the VT whether your map
applies to that.


Ranked
Ranked: If a map is ranked, that means it has been evaluated by the Ranking Team or RT. The RT judges each map submitted to them by the following criteria:
-The map must be judged to have enough quality to be official content representing osu!. This is determined by the RT.
-The map must adhere to the ranking standard, which the FTMT (Feature Team Management Team) will develop. This is mostly based on the osu! default ranking criteria.
-The map must be >44 seconds.

Featured

Featured: If a map is featured, that means it has been evaluated by the Feature Team or FT. The FT judges each map submitted to them by the following criteria:
-The map must be of outstanding quality.
-The map must offer value to mappers to be learned from.
-The map must show without a reasonable doubt that it is creative, logical, and made with incredible talent.


Team information (BN and QAT replacements)
The FTMT is the Feature Team Management Team. This team evaluates whether the FT has made correct choices in their features, and can un-feature a map. The VT is the Verification Team, the RT is the Ranking Team, and the FT is the Feature Team.

Each team will talk and make the ultimate decision. For any team to make a decision, they must come to a 51/49 decision. They will vote to either push the map forward or let it stay where it is at.

Each of the four teams can interact with the previous team (excluding the VT) by vetoing their decisions. For any team to veto, they must come to a 51/49 majority decision. A team cannot veto the team above them, or two tiers below them (i.e. the FTMT cannot veto the ranking team or the verification team).
After a veto, the map must address the issue that the vetoing team issued, and be resubmitted to the vetoing team. (If a featured map is vetoed, it becomes ranked again. Once the mapper fixes the issues, they will resubmit to the FT.)
If a map has a major timing issue and managed to get featured before anyone noticed, the FTMT may veto it, then the FT, then the RT will veto it and lastly, the VT will veto it. If this is the case, only the minimum and maximum team need to be submitted to. (In this case, the mapper would submit to the VT, and once they verify it, it can be submitted to the FT.)
Each team is elected in different ways:
The VT is elected by members of this server (who are any verified player) every month. There are 5 votes, and the top person in each vote will be elected into the VT. There are no term limits, but each member will retire if they are not a top voted person.

The RT is elected by the FTMT every 6 months. There are 5 votes, and the top person in each vote will be elected into the RT. There are no term limits, but each member will retire if they are not a top voted person.

The FT is elected by the FTMT every 8 months. There are 5 votes, and the top person in each vote will be elected into the FT. There are no term limits, but each member will retire if they are not a top voted person.

The FTMT is elected by mappers who have over 100 submitted maps and are regarded as a great mapper, every 12 months. There are 5 votes, and the top person in each vote will be elected into the FTMT. There are no term limits, but each member will retire if they are not a top voted person.


TL;DR: Three tiers, verified, ranked (like original), and featured. Verified means it's timed properly and finished, featured means it's extremely good. Four teams, each one evaluates the last team's decision. The fourth team evaluates the third but doesn't do anything else.

Please leave feedback below! It's appreciated.
xenal
Verified : sort of approved. Seems to exist only to reduce the amount of maps in graveyard. Splitting the goal of mappers isn't a good idea. Some will simply stop mapping toward ranked just because verified sounds less harder to reach since there isn't any standards. A lack of standard will also result in low quality maps being able to reach this state. Keep the mapping goal unified. This seems to encourage new mappers and give them a progression, however, ranked is currently an ultimate goal, you don't need a small step for scrubs.

Ranked : it's just ranked with RC being renamed and a change to minimum drain time

Featured : some sort of loved, but voted by a team of mappers instead of players. You say this is to target maps of high quality, however, as a little reminder that this category would exist 10 years from now on. Would you consider 2007 maps of outstanding quality nowdays? Maps of outstanding quality will be remembered and transfered from generation to generation through the game like it is currently, so a section for them will simply be a drama source. FT != everybody. Therefore, maps promoted by the FT will eventually be source of useless contreversy that the game doesn't need more.

Problem with current state of rank :
"If a featured map is vetoed, it becomes ranked again. Once the mapper fixes the issues, they will resubmit to the FT." This means being able to change .osu while map is ranked. Score and pp will be messed up if .osu if modified during ranked. If moved to unranked/verified or whatever, this still gives control of the leaderboard to the mapper, at the detriment of the players who maybe just had their top pp on that map, but now canceled because the mapper wanted featured. Qualified was invented just to fix that issue, your proposal is bringing it again.

Something that isn't said but I think you were thinking it (maybe?) is a progression, ranked maps need to be verified first and featured need to be ranked. At this point they then become : "I mapped the whole thing"->"My map fits standards"->"This ranked map is really good (for it's mapping era)". If so, verified sounds more like it already existed in a way in the past (thingy with the MAT).

VT and FTMT election system is biais, as public personnality affects result. Exemple "and are regarded as a great mapper". Some contreversial mappers will have a hard time to enter it, while they shouldn't have problem, simply because their maps brough them hate and love at the same time. While FTMT election can be done in closed group, like the current QAT members are chosen, VT is public.

VT : composed of random people, controlled by players and not mappers. They are also controlled by random people instead of responsables. In other words, people without mapping knowledge can control to some extends mapping. If such idea should be kept (which I don't like), it should be voted by mappers with 1+ ranked map/100+ kudosu.

Forming a voting system for EVERY decision is close to impossible. There will be too many requests, especially to the VT. Checking every single maps was removed from the requirements years ago (was it still BAT?) as it wasn't working out well, you want to bring it back (again something you bring back)

If FTMT is composed of all the great mappers, then the FT is composed of all the not-so-great mappers, then the RT is composed of the maybe-decent-but-not-great mappers and then VT is composed of that-one-mapper-Ive-seen-once-in-modreqs mapper. And if mappers are in multiple teams, attemps to prevent that veto thing you seems to like will happen. Mapping hierarchy that each control their section.

In summary : ranked is still ranked, verified splits mapping goals, featured is... I don't like, pp issues and team problems.

Your proposition is a mess of words, things are missing in there and you don't make every section clear of intentions. Changes just for changing will be widely rejected, so make that clear. In the end, in the current state of this proposal, I am all against.
abraker
I think this is breaking stuff into needless categories or tiers. What is this aiming to solve? Seems like a request suggesting something different without identifying why something like this is needed in the first place.

1) Why is the addition of the verified category needed? That sounds the same as the unraked maps you would find via sharing in multi. If you argue so that those unranked maps shared in multi have a group, then we have that already too. It's called loved.

2) Why is the additions of the Featured category needed? That sounds like ranked, but over the top. Either populated with maps that won a contest or maps that the FTMT decides to add there once in a blue moon, aka almost never. This category will be lucky to have more than a page by the end of the year in gamemodes other than standard.
dennischan
I kind of like the idea of verified. I'm thinking that it can be an automatic process, with BN, QAT, or anyone having over 3 ranked maps and 300 kudosu being able to verify maps. (150-200 kudosu for minigames). The automatic process is to reduce management effort and fit within the framework of modding v2.

The new ranking system can work like this:
Mapper submits map -> (Verified) -> Bubbled -> Qualified -> Ranked

Verification can act as a manual check which makes sure that there are no blatant unrankables and the map has at least a rankable full set. One verification from a BV (beatmap verifier) can verify the map. In my system verified would not give a leaderboard, so it incentivises people to continue going for rank.

Pros of my verified system
The pros of adding such a category is that it can signify an optional intermediate step in the ranking process, and help new mappers know that they are starting to get close to the goal of ranking a map.

Adding this intermediate step is good, because many new mappers never get even one BN to look at their maps. Adding one intermediate step will help new mappers know that they are improving, making them less likely to give up. Experienced mappers who do not need this step can continue ranking stuff without it being verified, so it wouldn't annoy experienced mappers.

Reply to OP
Edit: To reply to the op, I think that the system that your are suggesting is too convoluted to work. Reorganising the BN and QAT into 4 teams and making each team have to vote on ranking/featuring a map will end up being a large mess and people won't be able to rank/feature any maps at all. Voting on verified team is also stupid since the public has no sense of what a good map is.
clayton
some of these points are oddly specific for a general idea proposal, but I guess I'll be the first in this thread to say I actually like the idea a lot better than the current split of beatmap categories, particularly with the distinction of "Ranked" and "Featured" (though I think the names are misleading here)

currently:

Ranked gives maps leaderboards/pp, and maps are only Ranked when they meet certain quality standards
  1. Audiences:
    1. Players looking to compete with each other on nice maps
    2. Mappers looking for a sort of "approval" that their maps are high-quality and respected in the community, or looking for a Player audience
  2. Maintainer: Mappers only
Loved gives maps leaderboards, and maps are Loved if they see significant community support
  1. Audiences:
    1. Players looking to compete with each other maps that may or may not be "high-quality", but nevertheless fun to play
  2. Maintainer: Popular vote of Players

so IMO, there is a specific problem with the Ranked category: the audiences are not always the same as the people who maintain it. this means, inevitably, there will be some disconnect between the two groups. we can see this often enough today, with how some Ranked maps are strongly looked down upon by the general playerbase

and this is why I think this proposal could potentially build upon what we already have: it helps to move some of the current Ranked audience (mappers) to a category of their own that they can appreciate, being Featured. if I'm understanding this correctly it'd be like a showcase of the most creative and inspiring of maps for primarily other mappers, and it'd also act as a goal to achieve for aspiring creators. this would help alleviate the stress that the current Ranked section has to appeal to two groups that (usually) have conflicting views

not that there aren't also some significant problems with this though. I just have some general questions
  1. do Ranked and Featured overlap? or should they be strictly exclusive? can a map be Featured even if it doesn't meet Ranked criteria?
  2. what does each category actually do (like are leaderboards given, is pp allowed, anything else..?)
  3. by this system, most Loved maps would fit into Verified. that sounds like a downgrade for maps of community approval to be placed in the same category as Baby's First Map. should there be a different home for maps adored by players, but not by mappers? or is Verified okay?
  4. do you think Beatmap Spotlights already serve the same purpose as Featured? they are quite similar.

as-is this is kind of a big mess, but I think you're onto something here. I'm lead to believe the current issues of conflicting interests between "mappers" and "players" are structural and can be solved by re-arranging this Ranked system we've sort of just stuck with for 11 years

also I just want to note when I say "mappers" and "players" I don't mean to portray them as exclusive groups; nearly all mappers are players too. I think what I'm trying to convey is that "mappers" generally see more value in promoting creative and interesting map design, whereas "players" are more often just looking for playable things that they can enjoy without having to analyze the map critically

it's 5am and this post probably jumps all over the place so forgive me~
Topic Starter
Osuology
replying to xenal first: Verified is to encourage new mappers, and give players maps that are finished and timed properly. Nothing else. Quality should have NOTHING to do with verified in my opinion, the only goal is to provide finished and timed maps. I don't think splitting the goal of mappers is a bad idea, mapping always has its own progression regardless of the current system.
as for ranked, yes that is what it is for now, but that may change.
Featured would exist I think as a good way to look back at what mapping styles were like and what was considered good mapping for eras. You bring up that it would be eventually a source of useless controversy, but wouldn't ranking and loved be the exact same thing? People don't complain about how 2007 ranked maps got ranked, and if they do, those people are generally seen as ignorant of older times.

However, you bring up a very good point about PP and featured status. I do not have a solution to that, so that is a good point.
As well as the point about who are considered great mappers, I think I will change that to just mappers who have over 100 maps are allowed to vote. Maybe even a modding requirement as well.
The reason why the VT is voted by players, is because the verified section is for players. Not for mappers.
There quite possibly would be too many requests (but I do plan on having enough team members to take care of that. Maybe even upwards of 100 VT members.) But can't it be argued that the current system has that problem with BNs? There are nowhere near enough BNs for how many maps are trying to be ranked.
Thank you for your post! I made this late last night so it is definitely messy. Will work on it in the following days.

To reply to everyone else, the reasons why I want to change the system is as follows:
1. I think the current system is overflooded with maps trying to be ranked, and not enough BNs to follow up with it.
2. I think there is too much circlejerking across BNs, and too much lack of standards for ranked maps.
3. I think the current system could be much much more efficient with dealing with maps.
and 4. I think the teams could be handled much better and be much more clear about their decisions, and have a better hierarchy. I think the teams could be assembled better, by my proposals.

The criticism that this whole system is convoluted and may not work is certainly a possible one. I think it could work personally, if everything runs properly. And once again, the verified section is not about having good maps in my proposal, it's about having maps that are finished and timed properly, that's it.

dennischan, that's actually a more interesting and valuable setup for verified, and I may consider changing my proposal to that. Thank you.

And clayton, I agree with what you said about the Ranked category. Answering your questions:
Ranked and Featured in my original plan would overlap, so a featured map would have to be ranked. That may be changed though.
Leaderboards would be given to ranked and featured, I am not sure about PP. As the xenal pointed out, there is a glaring problem with allowing PP on the ranked maps in my system, so I'm still working on that. The categories are there to promote certain maps either as a playable map (verified), good map (ranked), or creative/amazing map (featured).
I think we could keep loved. I don't want to degrade loved maps to the level of verification.
I haven't seen much about beatmap spotlights, so perhaps I should look into that. Thanks!
abraker

Osuology wrote:

1. I think the current system is overflooded with maps trying to be ranked, and not enough BNs to follow up with it.
This is exclusive to the std gamemode. Mania barely has any maps being ranked right now, with periods where there are no maps in qualified at all. Please don't think stuff going on in standard applies to taiko, catch the beat, and mania because it does not.

Osuology wrote:

2. I think there is too much circlejerking across BNs, and too much lack of standards for ranked maps.
That is failure within the BN leadership. Making these categories will not fix this.

Osuology wrote:

3. I think the current system could be much much more efficient with dealing with maps.
Isn't this arguing against your proposal???

Osuology wrote:

4. I think the teams could be handled much better and be much more clear about their decisions, and have a better hierarchy. I think the teams could be assembled better, by my proposals.
This is not dependent on categorizing on what maps need to classified as. Some BN have their own agenda and beliefs on what maps should be in ranked that conflict with other BNs' beliefs and mappers' beliefs. Making these categories will not fix this issue. There will still be a ranked category and so this will still happen.

Osuology wrote:

And once again, the verified section is not about having good maps in my proposal, it's about having maps that are finished and timed properly, that's it.
This is what loved category is for
clayton

abraker wrote:

Osuology wrote:

And once again, the verified section is not about having good maps in my proposal, it's about having maps that are finished and timed properly, that's it.
This is what loved category is for

I manage Project Loved and I can safely say that is not at all what the Loved category is for
dennischan
Thanks to osuology for considering my proposal for verified. I suggested to have an automatic system for verified instead of a team because it would be hard to assemble a team of qualified people to mod maps in qualified, and it would essentially become BN lite and ranking lite if we had to pass through a hand picked team before reaching verified.

I thought of another problem in my proposal, and that is whether people can be allowed to skip the verified section. I would suggest people not being to able to skip the verified section, since if most experienced mappers skip the verified process, it would make it utterly meaningless as a checkpoint towards rank. And since the barrier to verified is so low, it wouldn't bring any trouble to an experienced mapper anyway.
abraker

clayton wrote:

I manage Project Loved and I can safely say that is not at all what the Loved category is for
Well ofc. The intent was to show that there is a place for maps Osuology is referring to already. I did not mean to say it's limited to just those kind of maps.

@Osuology concerning your first point...
Honestly, the ranked system is good as is. It works as intended. The problem are the people and logistics. A recommendation is to think about how logistics can be improved, because osu! had the MAT, BAT, QAT, BN, etc and all of them proved to have some kind of faults that does not allow a smooth ranking process 100% of the time and to the point that player/mappers/modders like you start to complain about its inefficiency. Think about how that process can be improved. The addition categories is not going to offset the strain ranked is getting. If you have people managing the additional categories, then this solution just splits the available workforce among the additional categories. The total throughput is still the same. If the total workforce will not increase, then the issue remains.

concerning your second point...
If there are lack of standards for ranked maps, then you submit proposals for ranked criteria here aiming to fix the lack of standards.

Last thing, verified as a category simply to allow spillage for maps not cut out for rank to go to, and that's fine. What I am not fine with is that it still needs to managed by a group, which will still lead to bottlenecks via logistics. Honestly, I think allowing every map to have a scoreboard and just let players set scores on them would solve 95% of all problems. If the map receives enough scores, then it can be automatically moved to a category where maps with many scores are found. Players would not play maps if they don't think they are good. Maps that are shit shouldn't be filled with many scores. Such system guarantees that, according to the playerbase, the map is worth playing if it's in that category. Call it automatic loved or something, but I see no reason why it can't work.
Topic Starter
Osuology
I definitely think the ranked category could still be better. The strain ranked is getting imo is because of the circlejerking, because of not much regulation on the BN team. I mean, obviously the strain increase comes from more players and more mappers, but then we should be adding more and more BNs and making the system more efficient. I think we have no where near enough BNs and not enough regulation on them to make them actually care about how many maps are trying to be ranked.
All in all, my system is kinda shit and that's probably because I came up with it late at night. There are good aspects to it still. I think the current system is fairly good, but I just really think that we need something big changed so that we can make this efficient and work for mappers, not for beatmap nominators. The purpose of the system is not so BNs can get their own maps ranked faster than any other mapper, the purpose should be that any good map gets ranked just as fast as any other good map can get ranked. It should be equal. But instead we see this huge backlog in the ranking process, and lots of BN circlejerking.
abraker
Prohibit BN from choosing which maps they process and force them to process maps at a certain rate or get kicked out. Guaranteed efficiency!

Or you know, take account that they are doing what they are doing on their spare time and have no actual obligation to do it. Take account they are remain interested in doing it because of the songs/maps they like, the friends they have, the quality the desire to uphold (for which every has their own perspective of).

Circlejerking will continue to be so long as people are allowed to say what they think quality should be, what they think the map should be. Differing opinions slow the process down a lot, and there is no solution to that other than making it automated in accordance to ranking criteria rules.
Please sign in to reply.

New reply