Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. osu!
  3. Gameplay & Rankings
  4. osu!mania

Difficulty System and PP Formula suggestions

posted
Total Posts
19
Shoegazer
osu!mania Paragon
528 posts
Joined April 2013
Topic Starter
Shoegazer
I would post the suggestions in the PP feedback thread, but I think there's too many things to cover and I don't really want to flood the entire thread with a single post, so I think it'd be better to make a thread for this.

The performance point system is admittedly, severely flawed - but there are ways to improve it well enough for it to be very accurate. Under this paragraph are two main things I think that could be improved to help the PP system. I admittedly haven't seen very much of the PP system, nor do I know much about the system at all - but I generally understand the fundamentals of it and how the fundamentals themselves could be improved. I'm not expecting these to be implemented or even read through, but it's probably better to give thoughts than to hide them and have zero chance for the PP system to improve.

Difficulty System
SPOILER
A little disclaimer, the examples I bring up and the terminology used only pertains to 4-key play, as it is the main mode I play and am most familiar with. However, the fundamentals of the difficulty system will apply to other keymodes as well, aside from 3K and under.

A huge factor of the PP distribution comes from the difficulty system, and is probably the main thing that should be fixed in order for the PP system to be fully useful. The main issues I've noticed regarding the difficulty system would be its overemphasis on maximum NPS and chords in general. AiAe is rated as the hardest 4k map by far, despite only being the second hardest ranked map overall based on personal experience and leaderboards. Compare it to Imperishable Night 2006, 1.52 stars (5.24 vs 6.76) LESS than AiAe, despite being harder overall. Having both of them at similar difficulty levels, if not slightly lower than IN2006 would be fine, but let's break down the factors of what makes a good difficulty system:

Speed
Speed is essentially split up into two different components: One hand trilling/gallops and jacking. The jacks in speed files are generally very brief, but the difficulty of jacks in streams go up far more exponentially than they do with one hand trills. That's why IN2006 is so hard, it's littered with tons of jacks in ultra-fast runningmen and with occasional one hand trills to boot.

However, fast bursts and speed in general are relatively low in NPS. A 450 BPM stream would be no faster than a 225 BPM jumptrill, or a 112.5 BPM quadjack. It's clearly obvious that the former is harder than the other two, and while the difficulty system does try to show a difference in difficulty between those, the differences are not major enough to differentiate between all of them accurately.

You can fix this by having a significantly lower weightage on max NPS and more weightage on patterning. One hand trills on any hand would increase the difficulty of a stream, even with it's just the minimum (3 notes) - which is why 1234321234 is harder than 123412341234. Jacks ramp up the difficulty as well, but it goes up exponentially IF the jacks that you're dealing with longer than 4-5 notes and you also have to hit other notes in a stream on the same hand. An example would be a runningmen, 43424342. I can't give numbers offhand on how much should a 300 BPM 10-note runningman (5-note 150 BPM jack in a stream) or a 260 BPM 6-note one hand trill should increase a difficulty of a map - but I'd say that a 300 BPM 12-note runningman is about as hard as hard as a 265-270 BPM one hand trill, so hidden jacks in runningmen should increase difficulty around 75%-80% more than a one hand trill with the same length. The longer the one hand trills/jacks in runningmen, the higher should be, of course.

As for hitting jacks while hitting other notes in a stream on the opposite hand, that belongs in the jacking category, which is what I'm going to talk about next:

Jacking
Due to the current difficulty system, chordsmash files are insanely overrated. Chordsmash files are especially overrated when it comes to fast walls, like AiAe's 180 BPM jumpjacks/360 BPM jumptrill - due to its emphasis on maximum NPS. There's too little emphasis on single jacking and jumpjacks that aren't necessarily overly fast, but changes columns often. How much difficulty should a single jack should be heavily determined by the speed and its length. An additional factor for jump/handjacks would be how often do the chords change direction.

For minijacks, the difficulty curve would be anti-exponential up to a point (like 350-380 BPM?), there isn't much of a difference between a 250 BPM minijack and a 300 BPM minijack, unless there's a bunch of them in one time, in that case the more one hand bias (e.g. 1122 or 3344) there is in the run of minijacks, the higher the difficulty should be. Gallop difficulty curve should be less exponential as one hand trills, but not considerably. The difficulty should also be higher when minijacks going into a hand or quad, as they are harder to transition into - especially with the lack of chord cohesion.

For jacks longer than 2 notes, the difficulty curves should be exponential - more exponential as the length of notes get longer up to a point (say 16 notes), the difficulty curve for 8-note jacks should start exploding by 155-160 BPM, but not as fast as it currently is at at the moment. When it comes to runningmen that has the jack on one hand and the rest of the runningman on the other hand, it should be rated higher than a single jack of course, but not by very much.

Regarding jumpjacks and handjacks, difficulty should be increased further if the direction of the chords change often. However, for jumpjacks, there should still be minijacks hidden inside the jumpgluts in order for the difficulty increase to count. In addition, jumpjacks that involve two hands (i.e. not [12] or [34] in a 4K context) should be rated higher than ones that involve one hand, but that's self-explanatory.

Fingers
This applies more towards 7K+ play, since there isn't too much of a difference in finger strength between the index and middle fingers due to human anatomy. Patterns that are more biased towards the outer fingers (e.g. the ring finger) and the thumb should be rated higher, especially one hand trilling and runningmen. The ring finger, pinky finger and thumbs are objectively worse for rhythm gaming, due to their lack of flexibility and strength. I heard that this will be implemented in the following difficulty system revamp however, so disregard this part if it is going to be implemented.

I didn't talk about noodles/long notes here, because I don't have that much experience or understanding on those to really give any constructive feedback on how it contributes to the difficulty system.

Ultimately, 5K+ maps will still be rated higher than 4K maps, even if both maps require the same amount of effort. This is because you have more fingers to deal with a general higher NPS, with more factors included in 5K+ play. This is inevitable, it's impossible to really encompass a completely fair difficulty system that covers every keymode. Because of this, and this will more than likely be a controversial opinion, will give a very clear bias towards 5K and up players. I'm not saying that 4K players are inherently better than 5K+ players by any means, but I feel that 5K+ players are rewarded too much for what they're actually doing. They already have an advantage over 4K players mainly because they could just use 1-3 less fingers to plow through 4K maps. You can argue that 4K and 6K+ have different emphasis in skills (5K is very similar to 4K), but if you're a top-tier 7K+ player, you should be able to plow through a good number of hard 4K maps. The reverse isn't true by any means, otherwise you'd see players like Staiain getting 96s and 97s in IN2006 7K Lunatic within a matter of weeks after destroying 4K. The fault is not in the difficulty system though, it's in the PP formula itself.

Performance Point Formula
SPOILER
The easiest way of reducing bias towards 5K+ players would be to reduce the amount of PP given for a map with more keys. It doesn't have to be much, the PP given doesn't even have to be the same as 4K players. It could be a general multiplier, or a exponential (albeit a very very small exponential curve) multiplier to adjust the amount of PP from a map with more keys accordingly. It would take some effort however, because you would have to check what would a 4K map be rated compared to a 5K/6K/7K/8K map if they have the same difficulty. With a perfect (or at least near-optimal) difficulty system however, an experienced o!m player (i.e. very good at every key mode) should be able to compare the difficulty between modes with relative ease. You could potentially create a multiplier from there.

How would this affect ranks overall? In order to be a player high up on the leaderboards, you would have to be proficient with every key mode. Even though you're really really good at 5K or 6K, if you're not good at 4K, you would not be high up on the leaderboards. 4K players would have to learn other key modes ultimately and it will take a longer process, but their ranks would not be understated as they were from before if this was added.

So that's pretty much it. There isn't much else I could talk about, since the PP formula and the difficulty system are the bread and butter of osu!mania's competitive gameplay in the grand scheme of things.

Would like to hear thoughts on this, counterpoints to these suggestions and improvements would definitely be appreciated.
Last edited by Shoegazer , edited 1 time in total.
Kamikaze
2,898 posts
Joined November 2012
Kamikaze
This is very interesting. I don't agree with "scaling pp for other keymodes" though. 7K will be biased anyway because it hasmuch more hard stuff ranked and good system won't have to scale anything to get accurate ratings. Big props for all the other stuff, I hope pp guys will read that many times
Shoegazer
osu!mania Paragon
528 posts
Joined April 2013
Topic Starter
Shoegazer

-Kamikaze- wrote:

This is very interesting. I don't agree with "scaling pp for other keymodes" though. 7K will be biased anyway because it hasmuch more hard stuff ranked and good system won't have to scale anything to get accurate ratings. Big props for all the other stuff, I hope pp guys will read that many times
Regarding the difficulty system: Of course, a perfect difficulty system would not have to scale for other keymodes - but that'd require tons of effort and while it is possible (completely different numbers for different keymodes, different factors involved), I can't quite imagine it happening any time soon. I guess you can see this as a way to fix the difficulty system as it is at the moment and an additional step to reduce bias towards higher keymodes, before revamping it to a difficulty system that is able to encompass all key modes accurately.

Anyway, you're right when you said that the 7K mappool has more hard maps than 4K (and other modes), and it will be biased because of that. Regardless though, in an ideal scenario where each mappool for each key mode has the same amount of hard maps, I do think it'll be completely unfair if the system gives significantly more PP to a player who plays 7K than one that plays 4K although both have the same score and both files require around the same amount of effort, because the difficulty system will on average higher-key maps higher than lower-key maps.

Thanks for the comment, much appreciated
lenpai
Beatmap Nominator
1,780 posts
Joined December 2014
lenpai
This is awesome. Talked about this in another game.
The problem is, formulating a difficulty analyzer if some sort would seem to be very difficult to make. The expounded patterns, LNs, and even SVs add up to a complex mix.
If you could give a game that has an accurate difficulty system (system/machine generated and not placed by people) i will stand corrected an also be surprised.
The 7k issue has been stated too.
Other than that there seems to be no problem with this.
Drace
314 posts
Joined April 2011
Drace
In my opinion, any sort of "finger placement dependent" criteria should be avoided. Many plays differently, whether it be with 2 thumbs or 2 pinkies or even 1 hand and 2 hands.

And while were at it, I doubt any kind of pattern sensitive algorithm would be able to satisfy the majority and overcome subjective difficulty. The best way to go about this, which will also be much easier to do, would be a dynamic system. A system that takes into account the performances and the general skill level of the people that plays it and uses it to determine the map's level.

To avoid data falsification we could say only players that has filled out their top plays slots, players with x-amount of plays or players which their ranking has stabilized are taken into consideration. Or really any filtering you see fit.

If no one in the top 100 can even S the map, the system will assign it an extremely high rating. If people from not even top 20k full combo it, it will receive an extremely low rating. Sure you'll get the random outliers that are better or worse at the patterns featured in the map, but the fact of the matter is the dynamic system will place this map at it's best possible place through data regression and averaging. This will overcome most of the subjective issues, the 7k/4k weighing and literally any other problems that come with deciding a difficulty for a map. The reason for this is it's based on real performance statistics and not subjective difficulty criteria.

The system was already put in place for LR2 as a third party project and really it works quite amazingly.
http://walkure.net/hakkyou/komakai.html FAQ and algorithm
http://walkure.net/hakkyou/bms.html The re-ranked charts based off clear type
lenpai
Beatmap Nominator
1,780 posts
Joined December 2014
lenpai
Considering the amount of players this should pull off well.
The stuff stated below won't be a possibility or o!m but I just want to state an example:
In the game where I came from (FtB) this was somehow practiced. In the early days most 1000 npm maps would recieve a 30/30 diff rating but as a few years passed by 1800 npm+ got the 30/30 rating and newer 1000 npms get 22-26/30 due to the increasing number of skilled players. (Patterns are taken into consideration, not just npm) Diff are votes by community so inactive voters dont get to revote.

However, in o!m, we have a very good spread of players from jhlee to n>1 star players and are also of good amount.
I don't see how this could fail. It's not like everyone will become an ET someday and that no beginners will remain.

This might also get to provide cleaner ratings (No diff will not exceed 10 stars where 10 stars is like the point of inpossibility or maybe 10.1 lol) *cough Fantazindy SHD
And yeah, since the diff will be demograhpically projected there should be no revoting problems.

All that is that needed is that new players should keep coming (not too frequently) because people improve in this kind of game. And if no new players arrive the average skill of all players increase. I've seen this happen to the point that (stating from previous example) N diffs will be like H diffs.
Bobbias
1,447 posts
Joined March 2011
Bobbias
Drace, the system would need a complete overhaul to implement something like that, and considering how the only big changes that happen are things peppy plans on doing years in advance (or at least it seems that way) I'd say there's very little chance of that happening. Changing difficulty means players can't necessarily predict how hard something will be just by looking at the number, making the entire difficulty rating system less immediately useful to players. A PP system based on changing difficulty values would also mean that everyone's PP would have to be recalculated each time the rating of a chart changed. If you got around that by finalizing the difficulty rating when a chart was ranked, that just makes the ratings even less useful for the player since the player would need to know when a map was ranked in order to know exactly what that difficulty value means. While this is less of a problem for players who keep up with the newest ranked maps, it becomes a big problem as time goes on and you get players with large collections of maps spanning multiple years.

I will admit that it would make the difficulty system much more accurate in providing actual difficulty values, but I don't think such a system will likely ever be incorporated into osu.

I agree that finger dependent pattern analysis is a no-go, because that makes assumptions about how players play, which is inherently bad when trying to make a system that judges the objective difficulty of a map.

IMO most of the 4k difficulty analysis in the first post here is essentially meaningless, because almost all of it relies on comparing specific patterns, and frankly falls under the same point I just made above: these all rely on the assumption that a player is using index and middle fingers on 2 hands separated equally, and that certain patterns are more or less difficult because of this. Sorry, but based on those assumptions alone most of your arguments on how to improve the difficulty algorithm aren't really useful.

While the difficulty system here is still not amazing, I think it's better than people give it credit for (and that's coming from someone who is really annoyed with how LNs are underrated compared to chord heavy charts).

Also, on the subject of adjusting for 5k+, it's an EXTREMELY bad idea to come up with some blanket unchanging way to adjust PP values down. That would effectively discriminate against players who focus on higher key count maps. If you were going to adjust PP because there are more or less ranked maps of a certain key count, you would need to individually adjust that based on the relative number of ranked maps of high difficulty. This would just incentivize high skill players to get scores on the difficult maps of whatever key count has the fewest hard charts ranked, which is something you should avoid.

I don't think it's a good idea to adjust the PP just because of that issue. Remember, mania still has a relatively small song collection. Be patient and a lot of these issues will eventually go away. It may take longer than we'd like, but I do think it will get better.
Shoegazer
osu!mania Paragon
528 posts
Joined April 2013
Topic Starter
Shoegazer
Lots to respond to:

Drace wrote:

In my opinion, any sort of "finger placement dependent" criteria should be avoided. Many plays differently, whether it be with 2 thumbs or 2 pinkies or even 1 hand and 2 hands.

And while were at it, I doubt any kind of pattern sensitive algorithm would be able to satisfy the majority and overcome subjective difficulty. The best way to go about this, which will also be much easier to do, would be a dynamic system. A system that takes into account the performances and the general skill level of the people that plays it and uses it to determine the map's level...
The BMS difficulty scale is definitely a beautiful piece of work, but it does bring up multiple issues if you're putting it into a game with multiple keymodes that has completely different players and skillset emphases:
- If you're basing it just on the player's ability to score on the map, maps with a very short (but significant enough to affect score by quite a bit) difficulty spike will be rated significantly higher than it should be. An example would be Spy's Bangin' Burst - the bursts are very short, but the bursts affect score relatively significantly. The difficulty of the file should increase because of that difficulty spike for sure, but it shouldn't be increasing anywhere as much as a difficulty formula through how everyone performs should.
- How would ability be assigned (if you're following the formula used in the BMS difficulty system)? They have different emphases and there are multiple files that test different skills, so it would probably be difficult to quantify all these scores into one solid value that is able to encompass both 4K+ play.
- It would require regular updating for every map, which would be a huge hassle years down the road when there are tons and tons of files that require updating. Unranked maps have no difficulty number in this case as well. Even if you create a leaderboard for unranked maps, it will more than likely have an inaccurate rating because the rating scale greatly relies on the number of people playing for an accurate rating; and most if not all unranked maps will have less plays than ranked ones.

I wouldn't mind seeing the system if the first and second points is looked at and there's a way to deal with the third point, however.

Bobbias wrote:

I agree that finger dependent pattern analysis is a no-go, because that makes assumptions about how players play, which is inherently bad when trying to make a system that judges the objective difficulty of a map.

IMO most of the 4k difficulty analysis in the first post here is essentially meaningless, because almost all of it relies on comparing specific patterns, and frankly falls under the same point I just made above: these all rely on the assumption that a player is using index and middle fingers on 2 hands separated equally, and that certain patterns are more or less difficult because of this. Sorry, but based on those assumptions alone most of your arguments on how to improve the difficulty algorithm aren't really useful.
This would only be true if using different fingers would cause a huge differentiation in pattern difficulty, which isn't the case. You can use your index-middle, your middle-ring, your ring-pinky, all of these patterns will still be difficult and the differences in difficulty between them when it comes to the patterns are minor enough for them to be insignificant.

This does lead to another question though - is there an objectively best finger placement? The index and middle fingers are the most flexible and they use the same nerve. I do think that the difficulty system should be based on the most optimal finger placement, and not by any other finger placement. It's further reinforced by the fact that index-middle, middle-ring and ring-pinky styles don't have too much of a difference other than overall performance of the same patterns - they're just worse in all aspects on an objective level, if you put in the same amount of practice into all of those styles.

I don't think there are many players who use 3 fingers on one hand and 1 finger on the other enough to be significant, not to mention it's far from an optimal playing style. You have too much emphasis on one hand, runningmen on the one hand with 3 fingers would be ridiculously hard, in turn you're able to hit runningmen on the one hand with 1 finger far more easily and be able to hit either 12121212 or 34343434 (depending on which hand with 3 fingers you used), which really isn't enough to counterbalance the disadvantages for me. It's a non-optimal style of play and I don't think non-optimal style of play should necessarily be rewarded more because they're doing well on a non-optimal style of play - because by the end of the day, the player with 3 fingers on the right hand and 1 finger on the left hand will find more maps harder than the given rating than maps they find easier than the given rating. It's not really a fault in the difficulty system, but moreso a fault in the style of play.

Bobbias wrote:

Also, on the subject of adjusting for 5k+, it's an EXTREMELY bad idea to come up with some blanket unchanging way to adjust PP values down. That would effectively discriminate against players who focus on higher key count maps. If you were going to adjust PP because there are more or less ranked maps of a certain key count, you would need to individually adjust that based on the relative number of ranked maps of high difficulty. This would just incentivize high skill players to get scores on the difficult maps of whatever key count has the fewest hard charts ranked, which is something you should avoid.

I don't think it's a good idea to adjust the PP just because of that issue. Remember, mania still has a relatively small song collection. Be patient and a lot of these issues will eventually go away. It may take longer than we'd like, but I do think it will get better.
That's not what I had in mind when I'm talking about scaling PP down. I'm taking about scaling it down because there's already an inherent bias, not because of the mappool, but because of the difficulty system used. If there are more hard maps in a 7K mappool than a 4K mappool and there are two players are equally skilled at 7K and 4K respectively with the scaling implemented, the 7K player will have more PP because there are more hard 7K maps than hard 4K maps. They just won't get as much as they would without the scaling.

The point of the performance point scaling is to reduce bias, not eliminate it completely.
Drace
314 posts
Joined April 2011
Drace
@Bobbias

You make it sound like difficulties will be bouncing from 2 star to 8 on a daily basis :P After the initial integration the map's difficulties will stabilize to a point that their difficulties won't change more than 0.01% a day due to the large population. Sure this will in turn cause the PP values to variate, but again it'd be nothing significant enough to bounce people around in the rankings. Every ups and downs that get done is simply because that small change was needed for a more accurate system. From a user perspective, it's still top players rank top slots on hardest songs. This in no way will cause any noticeable change to the current PP and difficulty system other than boosted accuracy.

The walkure integration is completely dynamic and I've yet to see any noticeable changes despite the scale going from level -1 to 27 at 2 decimal places. Feel free to keep it open in a tab and check on it daily. You'll only see like one or two charts moving by 0.01 every week. For there to be a noticeable change you'd require the entire population to suddenly start performing better on that one song (and not the others) in just a day.

I don't really see this causing an issue with determining a map's difficulty at a glance. Everyone has their own selection of maps they like, are familiar with and play often. Just from relativity they can easily see how much harder or easier an unknown map in any sort of scaling what so ever as long as it's accurate. And this proposed system will be offering a more accurate scale, so I'd say guessing a map's difficulty is actually more difficult in the current system if we were to compare. And again, the levels wont be jumping around like some mad experiment.

Also I wouldn't regard this as a "complete" overhaul since from a technical perspective, it's a system that can easily be made in parallel with the current one. And from a user perspective, everything will work the same except the maps will be be rated more accurately. PPv2 on the other hand was a complete overhaul and it happened.

Although I didn't particularly agree with the refutes you presented, I do have to say that there are complications that would arise from this system in other regards. More specifically the fact that this could only ever work on ranked maps. Not only does this mean all unranked maps will have to use an algorithm at best, we would need to put newly ranked maps on a "difficulty rating probation" to let their difficulty stabilize. Leaving players not knowing what exactly to expect on all newly ranked charts. (Which in my opinion isn't that bad at all, playing maps with no expectations is quite fun actually. Especially since sometimes you might perform better if you didn't know that map was rated to be harder than what you can normally handle :p) But I digress. Back on point, those rather small effects such a system will have is probably enough to have people disregards it's benefits and throw it in the bin.

Finally, I agree the odds of such implementations getting made here are pretty low, but you can't exactly deem it to be impossible ^^

@Psicicite

For starters, I'm not talking about the BMS table. The BMS scale runs of community voting. The walkure algorithm I referred to is a 3rd party project "for" BMS ^^ (there's like hundreds of third party stuff for BMS zz)

What the system will do is purposed to accomplish exactly what you're worried about; overcome difficulty biases. Regardless of what a map features (keymodes, difficulty spikes, patterns) it will use the entire playerbase's performances to come with a difficulty. Any method other than raw statistics will invite biases and even errors. It doesn't matter what is featured in the map since it's regressing the actual performances of everyone that plays it relative to other maps and the player's other performances.

And also it won't require any sort of re-updating, the system is automatic. But like I mentioned to Bobbias, the system wouldn't be compatible unranked maps. Nothing's stopping us from having user assigned difficulties or just the current algorithm for unranked maps, it's what all the other games are doing lol
Shoegazer
osu!mania Paragon
528 posts
Joined April 2013
Topic Starter
Shoegazer

Drace wrote:

@Psicicite

For starters, I'm not talking about the BMS table. The BMS scale runs of community voting. The walkure algorithm I referred to is a 3rd party project "for" BMS ^^ (there's like hundreds of third party stuff for BMS zz)

What the system will do is purposed to accomplish exactly what you're worried about; overcome difficulty biases. Regardless of what a map features (keymodes, difficulty spikes, patterns) it will use the entire playerbase's performances to come with a difficulty. Any method other than raw statistics will invite biases and even errors. It doesn't matter what is featured in the map since it's regressing the actual performances of everyone that plays it relative to other maps and the player's other performances.

And also it won't require any sort of re-updating, the system is automatic. But like I mentioned to Bobbias, the system wouldn't be compatible unranked maps. Nothing's stopping us from having user assigned difficulties or just the current algorithm for unranked maps, it's what all the other games are doing lol
My mistake, the walkure algorithm is a beautiful piece of work.

Anyway, that algorithm will definitely get rid of bias for sure, but the bolded part is really what I'm worried about. If you're basing it just on the players' performance (i.e. just the score), you are ignoring the fact that there are files that are harder to score on but not quite as structurally demanding. I guess the players at the lower end will struggle with the more structurally demanding map than the one that isn't but hard to score on, but I'm not quite sure how well the system would be to differentiate said difficulties. The algorithm would pretty much be perfect if it could differentiate maps like those well enough, given that it could already accurately determine difficulty of similar charts based on leaderboards perfectly.
[Ayase Eli]
42 posts
Joined May 2013
[Ayase Eli]
This stuff looks pretty cool. :o

Giant essay posts aside, I think it would be nice to see at least something added to mania ranking system, even if it is something small and overdue (coughcoughautoconvertscoughcough). CtB just saw a pretty sizeable change to its ranking system, maybe mania is next o.0
Kamikaze
2,898 posts
Joined November 2012
Kamikaze
Accorring to Tom changes aren't going to be applied very soon, but Blazier is currently more or less working on it, so changes will be there for sure. unrank autoconverts pls
Drace
314 posts
Joined April 2011
Drace
I don't really see the issue? If a song is actually harder on osu, it will be harder to score on osu. If a song is easy to score on, you can't just boost it's difficulty simply because there's this one part you find hard. Not only are you introducing bias, you're also rendering the whole difficulty scale obsolete by scaling it to mechanics that aren't featured in osu! and essentially giving players free PP.

I think you're thinking way too hard in this matter, the "main" purpose of the game is too score well. If it's harder to score on it, the map is simply harder. I get what you mean, under a different game following different mechanics, the said maps might indeed turn out to be harder. But were talking about osu! where we have score to represent your performance. For what purpose would you boost the difficulty rating on maps that are easy to score on over the ones that are harder? What are your standards to evaluate difficulty if not the scoring system? Game mechanics and "what ifs" that are completely irrelevant to this game?

You're saying you'd want to account for things that aren't apparent in the current scoring system within the difficulty algorithm. But I'd say it'd make much more sense to revamp the scoring system rather the difficulty scale. Or else both of them won't match with each other. The difficulty ratings should follow the scoring system, otherwise you're rendering one or the other obsolete.
Tidek
1,571 posts
Joined March 2011
Tidek
I just cant believe that AiAe is easier than in2006 for some people. I can almost S in2006 (need 0,5-1% more), but cant even pass AiAe :/
Shoegazer
osu!mania Paragon
528 posts
Joined April 2013
Topic Starter
Shoegazer
@Drace:
Oh wow, my phrasing is just absolutely horrendous in that one. I took a while to understand how that algorithm would apply, essentially what I wanted to ask is how would the algorithm deal with scoring curves (e.g. files like Bangin' Burst, where it is really hard to 99 but very easy to S, etc.) but I phrased it in such a brief and horrendous way that it was really misleading, my mistake.

Essentially I was curious on how well that algorithm would cover scoring curves like:
- As skill goes up, the score in a file will go up in a constant rate, maybe a little anti-exponential (because the room for error goes down exponentially) - until it reaches 100% - see AiAe for example
- As skill goes up, the score in a file will go up at a decreasing rate (gradient is lower as skill increases), up to 100% - an example of a map with that curve would be Bangin' Burst [Exhaust Lv. 16]
- As skill goes up, the score in a file will go up at an increasing rate (gradient is higher as skill increases), up to 100% - I don't know any ranked maps that are like that, but most fast jumpstream maps fit that criterion.

Then I realised that the algorithm would be able to cover it well anyway, since it covers the abilities of many players in leaderboards, not just the top.

I'm liking the algorithm more and more though, and I would like seeing it as the difficulty system instead, at least for ranked maps.
Drace
314 posts
Joined April 2011
Drace
Ah yes indeed that's more or less of a problem that would need to be thought of if this were to get integrated.

In the BMS integration, you can see 4 different grades. You have the easy clear, groove clear, hard clear and full combo relative difficulties. (To compare, it'd about A, S, 900k+, FC in osu). You can indeed take 2 charts and realize that although one chart is easier to groove clear, the other can still be easier to hard clear.

But in osu we don't really have diffrent "clear types" and we'd be forced to regress the map's difficulty into a single level. But this isn't really an unsolvable problem, it's just require a couple personal touches to fit this environment. A couple solutions I see would be:

- We could either pick 1 point, like getting 900k, and run the algorithm off that. (lazy way out, simple integration, not accurate on every other point)

- We could pick multiple points and regress a line through them. Sort of a more advanced averaging if you may. And then pick a point on the line. (more involved, still a fairly simple integration, although not perfectly "accurate" for every point; it will be the single most "precise" result we could get for a single dimension difficulty level).

- Or finally, we could actually revamp the system to allow the use of multiple points with sort of a "target" feature. The user would be able to set a target on their song selection UI, like 900k, and the maps will accurately and precisely display how hard they are to achieve 900k score. And on top of that the PP system could be scaled to these points for an even more accurate ranking system. (relatively much more work, more of a revamp than an integration, as accurate and precise that any sort of difficulty algorithm can possibly get).

---

Also back to a previous point about finger placement dependent difficulty. Most IIDX controller players play with (wrist)thumb/middle/index/index/thumb/middle/ring. Aka, thumb/middle finger on the first 2 lanes, not everyone plays symetrically :P
Last edited by Drace , edited 1 time in total.
ikzune
159 posts
Joined August 2013
ikzune
good ideas, i don't really have anything specific to add to this, ill let you all debate over what would work best because its hard, as kami said blazier is currently working on it but theres also this :P http://puu.sh/dJA2D/be8349f4d2.png
Tidek
1,571 posts
Joined March 2011
Tidek
Just remove autoconverts from pp system and it will be already a big upgrade in system.

Overall about pp system, things that need to be fixed is how star rating is calculating difficulty of a map, because: density = Star rating = more pp.

Like Psicicite said, star rating doesnt look on patterns but only on density of them (where its obvious that 12 34 trills are a lot easier than crossed 13 24 trills) and because of that we have maps that are over and under rated in star rating. (stamina is also a very important thing which should be considered in star rating).

And if we talking about personal experience, honestly in2006 has only one very hard pattern that makes ppl lose a lot of points, other parts of map are easier or the same hard like those AiAe quadstreams and I dont agree with that AiAe is easier than in2006 only because of 1 pattern, where AiAe requires a lot of stamina. (the same I can say that Haelequin is the hardest map because of extremely fast streams at the end which are mashable). But that its only my opinion and my personal experience from playing 4K for few years as well (just want to point out that some things can be easier or harder for other players).
Shoegazer
osu!mania Paragon
528 posts
Joined April 2013
Topic Starter
Shoegazer

Drace wrote:

Ah yes indeed that's more or less of a problem that would need to be thought of if this were to get integrated.

In the BMS integration, you can see 4 different grades. You have the easy clear, groove clear, hard clear and full combo relative difficulties. (To compare, it'd about A, S, 900k+, FC in osu). You can indeed take 2 charts and realize that although one chart is easier to groove clear, the other can still be easier to hard clear.

But in osu we don't really have diffrent "clear types" and we'd be forced to regress the map's difficulty into a single level. But this isn't really an unsolvable problem, it's just require a couple personal touches to fit this environment. A couple solutions I see would be:

- We could either pick 1 point, like getting 900k, and run the algorithm off that. (lazy way out, simple integration, not accurate on every other point)

- We could pick multiple points and regress a line through them. Sort of a more advanced averaging if you may. And then pick a point on the line. (more involved, still a fairly simple integration, although not perfectly "accurate" for every point; it will be the single most "precise" result we could get for a single dimension difficulty level).

- Or finally, we could actually revamp the system to allow the use of multiple points with sort of a "target" feature. The user would be able to set a target on their song selection UI, like 900k, and the maps will accurately and precisely display how hard they are to achieve 900k score. And on top of that the PP system could be scaled to these points for an even more accurate ranking system. (relatively much more work, more of a revamp than an integration, as accurate and precise that any sort of difficulty algorithm can possibly get).
I'd definitely say that the second point is the one I had in mind. I was thinking of maybe pass / 90% (maybe this is a little too close to a pass?) / 95% / 98.5% (maybe 99%, but 99% would not cover every map and hence will give a value of infinity for that difficulty, making it impossible to regress a line - but I might be thinking of this the wrong way). Not really a fan of using score in general because it's partially combo scoring and combo scoring is kinda gross.

The third point is probably possible, but yeah it's probably a little too much work I'd say.

Tidek wrote:

And if we talking about personal experience, honestly in2006 has only one very hard pattern that makes ppl lose a lot of points, other parts of map are easier or the same hard like those AiAe quadstreams and I dont agree with that AiAe is easier than in2006 only because of 1 pattern, where AiAe requires a lot of stamina. (the same I can say that Haelequin is the hardest map because of extremely fast streams at the end which are mashable). But that its only my opinion and my personal experience from playing 4K for few years as well (just want to point out that some things can be easier or harder for other players).
It's admittedly not really fair to compare AiAe to IN2006, in my post on scoring curves, AiAe would be in the first category and IN2006 would be in the second category. You're right that IN2006 only has one really really dumb part, but the 32nd bursts are still ridiculously hard to nail (or even combo without getting 100s/worse). I can definitely agree that AiAe easily requires more stamina than IN2006, but if you're top-tier, the stamina component could be ignored quite a bit (I personally don't get very tired in AiAe) and could breeze through a good portion of the file. Even if you're top-tier, the speed/control needed for IN2006 is still very prevalent because the patterns are that difficult. I guess it's better for me to say that IN2006 is harder to score on for top-tier players, but that'd be too long-winded. Haelequin's hard section is far too short and cheatable to be considered the hardest map.

(Before anyone talks about my scores on both songs; I could get mid 97s on AiAe on average (and SDCB'd AiAe on Stepmania), but I get low 97s/high 96s on IN2006 on average, my 97.90 is a fluke)

Regardless though, with that algorithm system AiAe should be a little higher than IN2006 I think.
Please sign in to reply.

New reply

0
1 / 19