Difficulty System and PP Formula suggestions

posted
Total Posts
19
show more
Drace
@Bobbias

You make it sound like difficulties will be bouncing from 2 star to 8 on a daily basis :P After the initial integration the map's difficulties will stabilize to a point that their difficulties won't change more than 0.01% a day due to the large population. Sure this will in turn cause the PP values to variate, but again it'd be nothing significant enough to bounce people around in the rankings. Every ups and downs that get done is simply because that small change was needed for a more accurate system. From a user perspective, it's still top players rank top slots on hardest songs. This in no way will cause any noticeable change to the current PP and difficulty system other than boosted accuracy.

The walkure integration is completely dynamic and I've yet to see any noticeable changes despite the scale going from level -1 to 27 at 2 decimal places. Feel free to keep it open in a tab and check on it daily. You'll only see like one or two charts moving by 0.01 every week. For there to be a noticeable change you'd require the entire population to suddenly start performing better on that one song (and not the others) in just a day.

I don't really see this causing an issue with determining a map's difficulty at a glance. Everyone has their own selection of maps they like, are familiar with and play often. Just from relativity they can easily see how much harder or easier an unknown map in any sort of scaling what so ever as long as it's accurate. And this proposed system will be offering a more accurate scale, so I'd say guessing a map's difficulty is actually more difficult in the current system if we were to compare. And again, the levels wont be jumping around like some mad experiment.

Also I wouldn't regard this as a "complete" overhaul since from a technical perspective, it's a system that can easily be made in parallel with the current one. And from a user perspective, everything will work the same except the maps will be be rated more accurately. PPv2 on the other hand was a complete overhaul and it happened.

Although I didn't particularly agree with the refutes you presented, I do have to say that there are complications that would arise from this system in other regards. More specifically the fact that this could only ever work on ranked maps. Not only does this mean all unranked maps will have to use an algorithm at best, we would need to put newly ranked maps on a "difficulty rating probation" to let their difficulty stabilize. Leaving players not knowing what exactly to expect on all newly ranked charts. (Which in my opinion isn't that bad at all, playing maps with no expectations is quite fun actually. Especially since sometimes you might perform better if you didn't know that map was rated to be harder than what you can normally handle :p) But I digress. Back on point, those rather small effects such a system will have is probably enough to have people disregards it's benefits and throw it in the bin.

Finally, I agree the odds of such implementations getting made here are pretty low, but you can't exactly deem it to be impossible ^^

@Psicicite

For starters, I'm not talking about the BMS table. The BMS scale runs of community voting. The walkure algorithm I referred to is a 3rd party project "for" BMS ^^ (there's like hundreds of third party stuff for BMS zz)

What the system will do is purposed to accomplish exactly what you're worried about; overcome difficulty biases. Regardless of what a map features (keymodes, difficulty spikes, patterns) it will use the entire playerbase's performances to come with a difficulty. Any method other than raw statistics will invite biases and even errors. It doesn't matter what is featured in the map since it's regressing the actual performances of everyone that plays it relative to other maps and the player's other performances.

And also it won't require any sort of re-updating, the system is automatic. But like I mentioned to Bobbias, the system wouldn't be compatible unranked maps. Nothing's stopping us from having user assigned difficulties or just the current algorithm for unranked maps, it's what all the other games are doing lol
Topic Starter
Shoegazer

Drace wrote:

@Psicicite

For starters, I'm not talking about the BMS table. The BMS scale runs of community voting. The walkure algorithm I referred to is a 3rd party project "for" BMS ^^ (there's like hundreds of third party stuff for BMS zz)

What the system will do is purposed to accomplish exactly what you're worried about; overcome difficulty biases. Regardless of what a map features (keymodes, difficulty spikes, patterns) it will use the entire playerbase's performances to come with a difficulty. Any method other than raw statistics will invite biases and even errors. It doesn't matter what is featured in the map since it's regressing the actual performances of everyone that plays it relative to other maps and the player's other performances.

And also it won't require any sort of re-updating, the system is automatic. But like I mentioned to Bobbias, the system wouldn't be compatible unranked maps. Nothing's stopping us from having user assigned difficulties or just the current algorithm for unranked maps, it's what all the other games are doing lol
My mistake, the walkure algorithm is a beautiful piece of work.

Anyway, that algorithm will definitely get rid of bias for sure, but the bolded part is really what I'm worried about. If you're basing it just on the players' performance (i.e. just the score), you are ignoring the fact that there are files that are harder to score on but not quite as structurally demanding. I guess the players at the lower end will struggle with the more structurally demanding map than the one that isn't but hard to score on, but I'm not quite sure how well the system would be to differentiate said difficulties. The algorithm would pretty much be perfect if it could differentiate maps like those well enough, given that it could already accurately determine difficulty of similar charts based on leaderboards perfectly.
[Ayase Eli]
This stuff looks pretty cool. :o

Giant essay posts aside, I think it would be nice to see at least something added to mania ranking system, even if it is something small and overdue (coughcoughautoconvertscoughcough). CtB just saw a pretty sizeable change to its ranking system, maybe mania is next o.0
Kamikaze
Accorring to Tom changes aren't going to be applied very soon, but Blazier is currently more or less working on it, so changes will be there for sure. unrank autoconverts pls
Drace
I don't really see the issue? If a song is actually harder on osu, it will be harder to score on osu. If a song is easy to score on, you can't just boost it's difficulty simply because there's this one part you find hard. Not only are you introducing bias, you're also rendering the whole difficulty scale obsolete by scaling it to mechanics that aren't featured in osu! and essentially giving players free PP.

I think you're thinking way too hard in this matter, the "main" purpose of the game is too score well. If it's harder to score on it, the map is simply harder. I get what you mean, under a different game following different mechanics, the said maps might indeed turn out to be harder. But were talking about osu! where we have score to represent your performance. For what purpose would you boost the difficulty rating on maps that are easy to score on over the ones that are harder? What are your standards to evaluate difficulty if not the scoring system? Game mechanics and "what ifs" that are completely irrelevant to this game?

You're saying you'd want to account for things that aren't apparent in the current scoring system within the difficulty algorithm. But I'd say it'd make much more sense to revamp the scoring system rather the difficulty scale. Or else both of them won't match with each other. The difficulty ratings should follow the scoring system, otherwise you're rendering one or the other obsolete.
Tidek
I just cant believe that AiAe is easier than in2006 for some people. I can almost S in2006 (need 0,5-1% more), but cant even pass AiAe :/
Topic Starter
Shoegazer
@Drace:
Oh wow, my phrasing is just absolutely horrendous in that one. I took a while to understand how that algorithm would apply, essentially what I wanted to ask is how would the algorithm deal with scoring curves (e.g. files like Bangin' Burst, where it is really hard to 99 but very easy to S, etc.) but I phrased it in such a brief and horrendous way that it was really misleading, my mistake.

Essentially I was curious on how well that algorithm would cover scoring curves like:
- As skill goes up, the score in a file will go up in a constant rate, maybe a little anti-exponential (because the room for error goes down exponentially) - until it reaches 100% - see AiAe for example
- As skill goes up, the score in a file will go up at a decreasing rate (gradient is lower as skill increases), up to 100% - an example of a map with that curve would be Bangin' Burst [Exhaust Lv. 16]
- As skill goes up, the score in a file will go up at an increasing rate (gradient is higher as skill increases), up to 100% - I don't know any ranked maps that are like that, but most fast jumpstream maps fit that criterion.

Then I realised that the algorithm would be able to cover it well anyway, since it covers the abilities of many players in leaderboards, not just the top.

I'm liking the algorithm more and more though, and I would like seeing it as the difficulty system instead, at least for ranked maps.
Drace
Ah yes indeed that's more or less of a problem that would need to be thought of if this were to get integrated.

In the BMS integration, you can see 4 different grades. You have the easy clear, groove clear, hard clear and full combo relative difficulties. (To compare, it'd about A, S, 900k+, FC in osu). You can indeed take 2 charts and realize that although one chart is easier to groove clear, the other can still be easier to hard clear.

But in osu we don't really have diffrent "clear types" and we'd be forced to regress the map's difficulty into a single level. But this isn't really an unsolvable problem, it's just require a couple personal touches to fit this environment. A couple solutions I see would be:

- We could either pick 1 point, like getting 900k, and run the algorithm off that. (lazy way out, simple integration, not accurate on every other point)

- We could pick multiple points and regress a line through them. Sort of a more advanced averaging if you may. And then pick a point on the line. (more involved, still a fairly simple integration, although not perfectly "accurate" for every point; it will be the single most "precise" result we could get for a single dimension difficulty level).

- Or finally, we could actually revamp the system to allow the use of multiple points with sort of a "target" feature. The user would be able to set a target on their song selection UI, like 900k, and the maps will accurately and precisely display how hard they are to achieve 900k score. And on top of that the PP system could be scaled to these points for an even more accurate ranking system. (relatively much more work, more of a revamp than an integration, as accurate and precise that any sort of difficulty algorithm can possibly get).

---

Also back to a previous point about finger placement dependent difficulty. Most IIDX controller players play with (wrist)thumb/middle/index/index/thumb/middle/ring. Aka, thumb/middle finger on the first 2 lanes, not everyone plays symetrically :P
ikzune
good ideas, i don't really have anything specific to add to this, ill let you all debate over what would work best because its hard, as kami said blazier is currently working on it but theres also this :Phttp://puu.sh/dJA2D/be8349f4d2.png
Tidek
Just remove autoconverts from pp system and it will be already a big upgrade in system.

Overall about pp system, things that need to be fixed is how star rating is calculating difficulty of a map, because: density = Star rating = more pp.

Like Psicicite said, star rating doesnt look on patterns but only on density of them (where its obvious that 12 34 trills are a lot easier than crossed 13 24 trills) and because of that we have maps that are over and under rated in star rating. (stamina is also a very important thing which should be considered in star rating).

And if we talking about personal experience, honestly in2006 has only one very hard pattern that makes ppl lose a lot of points, other parts of map are easier or the same hard like those AiAe quadstreams and I dont agree with that AiAe is easier than in2006 only because of 1 pattern, where AiAe requires a lot of stamina. (the same I can say that Haelequin is the hardest map because of extremely fast streams at the end which are mashable). But that its only my opinion and my personal experience from playing 4K for few years as well (just want to point out that some things can be easier or harder for other players).
Topic Starter
Shoegazer

Drace wrote:

Ah yes indeed that's more or less of a problem that would need to be thought of if this were to get integrated.

In the BMS integration, you can see 4 different grades. You have the easy clear, groove clear, hard clear and full combo relative difficulties. (To compare, it'd about A, S, 900k+, FC in osu). You can indeed take 2 charts and realize that although one chart is easier to groove clear, the other can still be easier to hard clear.

But in osu we don't really have diffrent "clear types" and we'd be forced to regress the map's difficulty into a single level. But this isn't really an unsolvable problem, it's just require a couple personal touches to fit this environment. A couple solutions I see would be:

- We could either pick 1 point, like getting 900k, and run the algorithm off that. (lazy way out, simple integration, not accurate on every other point)

- We could pick multiple points and regress a line through them. Sort of a more advanced averaging if you may. And then pick a point on the line. (more involved, still a fairly simple integration, although not perfectly "accurate" for every point; it will be the single most "precise" result we could get for a single dimension difficulty level).


- Or finally, we could actually revamp the system to allow the use of multiple points with sort of a "target" feature. The user would be able to set a target on their song selection UI, like 900k, and the maps will accurately and precisely display how hard they are to achieve 900k score. And on top of that the PP system could be scaled to these points for an even more accurate ranking system. (relatively much more work, more of a revamp than an integration, as accurate and precise that any sort of difficulty algorithm can possibly get).
I'd definitely say that the second point is the one I had in mind. I was thinking of maybe pass / 90% (maybe this is a little too close to a pass?) / 95% / 98.5% (maybe 99%, but 99% would not cover every map and hence will give a value of infinity for that difficulty, making it impossible to regress a line - but I might be thinking of this the wrong way). Not really a fan of using score in general because it's partially combo scoring and combo scoring is kinda gross.

The third point is probably possible, but yeah it's probably a little too much work I'd say.

Tidek wrote:

And if we talking about personal experience, honestly in2006 has only one very hard pattern that makes ppl lose a lot of points, other parts of map are easier or the same hard like those AiAe quadstreams and I dont agree with that AiAe is easier than in2006 only because of 1 pattern, where AiAe requires a lot of stamina. (the same I can say that Haelequin is the hardest map because of extremely fast streams at the end which are mashable). But that its only my opinion and my personal experience from playing 4K for few years as well (just want to point out that some things can be easier or harder for other players).
It's admittedly not really fair to compare AiAe to IN2006, in my post on scoring curves, AiAe would be in the first category and IN2006 would be in the second category. You're right that IN2006 only has one really really dumb part, but the 32nd bursts are still ridiculously hard to nail (or even combo without getting 100s/worse). I can definitely agree that AiAe easily requires more stamina than IN2006, but if you're top-tier, the stamina component could be ignored quite a bit (I personally don't get very tired in AiAe) and could breeze through a good portion of the file. Even if you're top-tier, the speed/control needed for IN2006 is still very prevalent because the patterns are that difficult. I guess it's better for me to say that IN2006 is harder to score on for top-tier players, but that'd be too long-winded. Haelequin's hard section is far too short and cheatable to be considered the hardest map.

(Before anyone talks about my scores on both songs; I could get mid 97s on AiAe on average (and SDCB'd AiAe on Stepmania), but I get low 97s/high 96s on IN2006 on average, my 97.90 is a fluke)

Regardless though, with that algorithm system AiAe should be a little higher than IN2006 I think.
Please sign in to reply.

New reply