So I got inspired by iJinjin to do some of my osu! research. I was going for PP by starting examining my stats and then a highly ranked player. That was what I going to do at first, but got sidetracked into determining ACT's powe-I mean star level. I don't expect this to be entirely accurate, but ACT, do reply on how well I predicted. So here are my findings:

- - - - - - - - - - - - - - - - - - - - - - - - - - - [WARNING: Some math involved] - - - - - - - - - - - - - - - - - - - - - - - - - -

Refer to the spreadsheet:
Taken from his top ranks, on a scale if the max score was to be equal to star count*1,000,000, we are yet to see the extent of ACT's abilities. While a minor degradation is visible towards 7 stars, in order to find his degradation we will need to find a point in which the score gained on this scale starts to decline more rapidly the score he gets at a star level. First lets set the slope relative to the maximum score possible to achieve at a certain star level (1 star: 1,000,000, 3 star: 3,000,000, etc), giving as a slope of 1,000,000.

If we take a look at the "Star vs Score" graph, the generated line of best fit needs to be adjusted to the the scale the other graph uses. We can do this simply be multiplying the formula by X. Then we get this graph (used: https://www.desmos.com/calculator). I am not sure why, but I need to add 500,000 to this graph to fix offset:


This graph should not be taken as is, but it seems to correlate. According to the graph, ACT's peek performance is on a 6.443 star and his limit seems to be almost 9 stars O_o. There is a noticeable linear slope from 1.741 stars to 4.98 stars, and that represents the star levels he can most likely SS. The farther away from the line, the less likely an SS is going to happen. I don't think I need to go over what the graph means for maps below 1.0 stars...

Procedure:

• 1) Make and fill in 2 column table in Exel that will list the star level and the player score
  2) Select the entire star level column followed by the score column
  3) Make a scatter plot graph
  4) Right click on one of the points and add a 3rd order polynomial trendline. Click show formula.
  5) Copy the formula, go to https://www.desmos.com/calculator and paste it as the first function. Correct the formatting. 1+E6 is 1 followed by 6 zeros (1 million), and 4+E6 is 4 followed by 6 zeros (4 million), etc
  5) Click at the wrench (top right corner), and fill in the X and Y ranges as you see in the graph from Exel. Adjust the values as needed.
  6) If the graph has the right shape but the wrong values, go to Exel, right click on the label, and select "Format Trend Line Label". Under "catagory" select number, and set the decimal places to something higher (something like 4). Goto step 5.
  7) Copy the first formula, paste it as the second formula, and multiply it by X.
  8) Adjust ranges as needed.
  9) To compare skill level graphs, normalize the individual graphs by dividing each respective graph by its max value.

Here is the linked spreadsheet, where you might find a bit more stuff I played with.
https://www.dropbox.com/s/g1qbiwmkd8ol5bf/research.xlsx?dl=0

If there is a high demand, I might get motivated in making a tool that auto calculates this for any player. Don't know yet.

[4/24/15] EDIT: Updated spreadsheet to include Tristan97's data

Nice graph and analysis

now get ACT to see this hue

So you saw the skill level of a highly skilled player, but how does the skill level for an average player like me look like? After figuring out why the trend line was wrong in Exel (number precision, argh), you can now see. For a player like me who's unstable rate in accuracy is as high as playing while being... high, it looks like a roller coaster.


Immediately you see that my peak is 1.3 stars. While the performance table doesn't go below 3 stars, I was very surprised that it got that right. Where do you think all my 700+ SS's come from? Next you notice a dip in skill, having a depression at 3.5 stars. That kinda surprised me and kinda didn't, predicting a score of about 780,000. The only way a can explain this is that hold notes and streams at this difficulty are harder for me than jumping beats at 4 star maps. Perhaps I can do better on those.

Next we see a second peak at 4.5 stars. While not as high as the 1.3 stars's peak, it definitely stands out. Now what you have to realize is that it doesn't mean my score in that is better than on a 3.5 star. It means that considering the difficulty level to score ratio, I do 4.5 stars better than 3.5 stars. Our final point is at about 5.2 stars. I marked that because as it says, that's the hardest map I B'd, being Back To The Gate from the Yolomania collection. That said, I easily fail anything above 4.8 stars.

Here is the normalized graphs of skill level comparing me to ACT. This should go without saying, ACT would beat me on maps below 4 stars even if the graph shows otherwise. I will need to figure out a good way to prevent the graph from dropping as star level approaches to 0.

I am still researching about player's skill levels and will do top players from other modes next to figure out how the skill distribution in those modes is like:
WubWoofWolf - Standard
ExGon - CtB
??? - Taiko (haven't decided)

If you want to be analyzed, I want to analyze players ranked close to 300, 1000, 7000, and 10000 from each of the modes (making 4 players from each mode). If you want to participate, PM me. In doing this, I am hoping to come up with a good way (formula) to compare players' skill level in the feature request I have: t/318011.

Available Slots: (red means that a candidate exists, and crossed means position taken)
Mania · · · · · · · · · · · CtB· · · · · · · · · · · · · · Taiko· · · · · · · · · · · · · · · Osu
#300· · · · · · · · · · · ·#300· · · · · · · · · · · · · #300· · · · · · · · · · · · · · ·#300
#1000· · · · · · · · · · · #1000· · · · · · · · · · · · #1000· · · · · · · · · · · · · ·#1000
#7000· · · · · · · · · · · #7000· · · · · · · · · · · · #7000· · · · · · · · · · · · · ·#7000
#10000· · · · · · · · · · #10000· · · · · · · · · · · ·#10000· · · · · · · · · · · · ·#10000

can you explain this to me in stupid level?

CptBlackBird wrote:

can you explain this to me in stupid level?

Look at the graph.

The numbers at the bottom is the star level.

The numbers on the left represents skill level.

The arrow pointing to "Performance Graph" is how the skill for the star levels looks like.

The arrow pointing to "Score Graph" is how the scores for the star levels looks like.

The closer the "Performance Graph" is to the straight line that has an arrow pointing "Confidence" to it, the higher chance there is for an SS at that star level.

That's all.

thanks

Oh I'd definitely be interested in being the ~#1000 rank case study for osu!mania

I was hoping there would be an simple way to figure this out, analysis like this is really intriguing to me.

(I've played every ranked osu!mania beatmap, so hopefully that can help).

If you need another person just under 1k, I'd be interested too. I've actually thought about doing something like this myself, but I never knew where to start with the analysis.

If Tristan97 doesn't respond to my PM within the next day, you will be the next candidate for #1000 rank mania. Be on a lookout for a PM.

Edit: Rank #1000 mania category is taken.

Using a cubic polynomial for fitting the data doesn't seem to work well for all cases.

For example, this is how the curves look for my top best performances with cubic fit:
With that fit, difficulties higher than what I can play have a higher predicted score than the ones I can play.


This using a fit of the form instead.
(Note: this is the same as fitting for , but using the variable c makes Mathematica's default algorithms more numerically stable)

Extra: Abraker's graph with the other fitting.
jhlee0133's graph
F1stLove's graph

Oh interesting, will investigate later today. While your methods fits better, my method seems to have been less accurate but showed some accuracy on a wider range overall for the cases I tried.Also both of our methods need fixing toward the lower end of the spectrum. (yours actually works!) Can you share the list of the scores you used to make the graph of your skill distribution?

Full Tablet wrote:

<jhlee0133's graph>

HOLY FIRETRUCK!!! Still strong at 10 stars?!?

jhlee is a beast:

https://www.youtube.com/watch?v=RVXx5LH4lIA

https://www.youtube.com/watch?v=Xro40rys6Vs

abraker wrote:

Can you share the list of the scores you used to make the graph of your skill distribution?

Here they are in an excel document http://a.pomf.se/icupib.xls

You can use the Solver function in excel to make a fit of the data using other functions (I haven't tried it that way though, I used Mathematica), the end of this guide has some instructions for this http://www.math.ksu.edu/math100studio/f ... el2007.pdf
(I used the a function that mathematically should give the same result than the one used in the guide, but I set "c" to 1,000,000 instead of fitting for it)

You should have used all ~100 records instead of ~50 for better accuracy. Other than that, I am surprised it generates a positive curve instead of a negative one, which is what is throwing off the prediction. Perhaps it is due to that your scores are all over the place, which would pose a problem if predicting even more unstable data. Can you try entering the rest of the scores?

Full Tablet wrote:

the end of this guide has some instructions for this

Thanks for the the resource!

Full Tablet wrote:

You can use the Solver function in excel to make a fit of the data using other functions (I haven't tried it that way though, I used Mathematica),

It doesn't generate an equation nor a graph. It solves for a value.

I will be adding more to this post after the weekend. I can't find a tool the generates the best fit based on a logistics function (not looked hard enough?), and even if there is one, it would be interesting to know how it works. I will be experimenting in code and consulting the professors at my college in the meantime. If you guys have anything to add to this, that would be great! Here is what I got so far (incomplete):

I tried to model Tristan97's data using the 3rd order polyomial and it failed like FullTablet's. It doesn't seem to do well with highly unstable data. So I decided to try FullTablet's suggestion. Since Exel doesn't provide the formula in the form that he provided and I don't have Mathematica, this became a quite trivial task.

Using the Wiki, I found that the coefficients/constant values in the function have these meanings (using FullTablet's naming of the coefficients):


Logistic Function:
a = max slope/4
b = x value at which the slope is a
1,000,000 = max value of the graph
C = shifts the b value by log(C)/a units to the left

That's nice and all, but it doesn't tell us how to model the function according to the data. If you take a look at FullTablet's pics, the only part that we have that we can go by is at the peak or just before the peak.

Resources that would require me sometime to figure out:
http://www.cs.xu.edu/math/math120/01f/logistic.pdf
http://socserv.mcmaster.ca/jfox/Books/Companion/appendix/Appendix-Nonlinear-Regression.pdf
http://home2.fvcc.edu/~dhicketh/DiffEqns/Activities/logistic.pdf
http://stats.areppim.com/glossaire/scurve_def.htm <---- Looks most useful
http://wmueller.com/precalculus/families/1_82.html

[MORE TO BE ADDED]

I am #474. Am I close to #300?
My results vary widely so this might as well break all these calculations lol
Any tool for gathering needed data for this? Doing this by hand is pretty tiring

You were #375 at one point, so I'll give you that. I don't think there is an easy way to this (I'll come up with something hopefully soon). Just list the star level followed by the score in a text file of all 100 maps listed in your top ranks, upload it and link here. I'll mark you as a candidate.

To Mods: This is becoming less of a mania thing, and more of a general thing. Mind moving this to the appropriate section?

abraker wrote:

I tried to model Tristan97's data using the 3rd order polynomial and it failed like FullTablet's. It doesn't seem to do well with highly unstable data. So I decided to try FullTablet's suggestion. Since Excel doesn't provide the formula in the form that he provided and I don't have Mathematica, this became a quite trivial task.

You can use the Solver feature in excel for making a fit of the data with the instructions given in the guide (I tried it out, and ended up with the same results than with Mathematica). You end up with values for the constants "a" and "b" that make the logistic function fit the data the best (the parameter "c" is not needed, since all it does is shifting the graph the same way b does), then you can plot the function with the parameters in another application.

Kert wrote:

Any tool for gathering needed data for this? Doing this by hand is pretty tiring

You can use the osu! API and a JSON parser to retrieve data quickly (the limitation is that you can only get information about the top 50 best performances for each player, instead of 100 as when doing it manually or by writing a program to parse information of the osu! website).

So you guys don't feel like this is falling silent, I am making a making a program to do this.

Full Tablet wrote:

You can use the osu! API and a JSON parser to retrieve data quickly (the limitation is that you can only get information about the top 50 best performances for each player, instead of 100 as when doing it manually or by writing a program to parse information of the osu! website).

Finished this part. Next is the hard part, which is determining how to generate the equation of the graph.

Full Tablet wrote:

You can use the Solver feature in excel for making a fit of the data with the instructions given in the guide

I'll need more than that to pull off what I'm trying to accomplish now.

I still have many positions open for the case study/research. Refer to this post on what positions are open.