The struggles…

What is the best way to get better at osu?

This question has haunted many tens of thousands of players over the years. Lots of time was spent on it in an effort to find an answer, many things have been tried and many ideas have been discarded. At some point the community agreed on certain core pieces of advice, as I present them to you here.



The 3 tenets of improvement in osu


I. Play more

II. Push your limits with hard maps

III. Play a great variety of maps and don't spam retry


These tenets seem self-evident, most people that follow them expect to benefit from them greatly, and it feels like they’ve been around forever. I suspect that one of these tenets doesn't belong on the list: The 3rd rule of improvement might turn out to be a sham.
Why do I think that? Once we're at the end of this thread, it will be clear.


_________________________________________________________________________________


Let me start off by saying that the „never retry“-rule hasn’t actually been around forever. If you go back 4-5 years and check the two most popular guides for improvement, "How to improve at osu! (by jesse1412, 400.000 views)" and "Why you aren’t improving at osu! HOW DO I GET BETTER?!?“ (by Scarletstory, 210.000 views) . Even though one of them mentions that playing a great variety of maps is benefitial, you’ll see that neither of them mention anything about how often you are supposed retry.

So where did this belief originate? I don’t know if he was the first to come up with it, but I do know that he was the first to popularize it, spreading it far and wide:



Rohulk on improvement, his ask.fm almost 2 years ago

Since then he’s been talking about it regularly on his streams, different threads made about improvement frequently include advice about not retrying, and other top players started recommending the same thing.

So... does that make sense?
Here is the logic behind Rohulk's idea as I understand it: It’s better to play many different maps because you’ll get to experience more variation. Not only do you get more consistent, but you'll also improve faster.
If you retry a lot and keep playing the same parts oft he map, you’ll improve slower and you'll be less consistent because you don’t get to live through all of the variety that you’d otherwise have.


That’s a cool theory and it does seem to make sense at face value. This theory predicts that people who retry the same maps a lot would improve way slower.

But stop for a second and look at this instead:

CXu wrote:

I don't know why there's this mantra about never retrying maps. If you play a map, do something wrong, then you should also know or figure out what you did wrong, and then fix it. If you keep playing different maps, you're just letting whatever you did wrong stay in your muscle memory.

source


Well, now we have a real problem, what CXu said also seems to make a lot of sense! Maybe instead of playing lots of variety, we should focus on fewer maps and perfect certain patterns?


In short: we have two conflicting theories that predict different things.

- According to Rohulk's logic we should retry as little as possible and play lots of different maps to get better.

- According to CXu's logic we should exactly not do that, but instead play the same stuff again and again to fix mistakes we might be making.



Who is right and who is wrong? Are both wrong? Maybe both are right to some degree? What is going on?

And this is the problem with mere conjecture and biased self-reports: There is no real way to tell. So let's try something else instead and look at some data.

at this point I should be thanking abraker, who made this possible by writing the program that fetched the much needed data. Thanks a lot buddy! And it only took what felt like 15 minutes. You're a real beast.

After about 9 hours of running the program on the top10k in osu, we ended up with ... a crapload of data. Rank, pp, playcount, hitcount and other categories are now sitting on my computer in an excel chart that extends 10.000 rows down.
Thanks again, abraker!



So let's get started on the data. We're plotting (pp/hitcount) against (hitcount/playcount).
pp/hitcount ≈ rate of improvement
hitcount/playcount = average play length ≈ retry-rate



_________________________________________________________________________________




Every single player of the top10k is a dot on this chart


Let's explain this briefly:

- The further you are up on the chart, the faster you improve. The people that are WAY above everyone else are mostly hackers, multi-accounts, people who played offline their entire lives or touchscreen players who exploit the pp-system (the data is pre-touchscreen-nerf).

- The further to the right you are, the less you retry. The dot that is way to the right is actually Rohulk himself, so congrats to him for sticking to his principles!

The dots are clumped together pretty badly, so let's see if we can take a closer look..



enhance!

it seems to unravel after a rate of improvement >0.0015. Everyone past that point gets treated as an outlier, the people that have "something weird going on with their account".
Cookiezi is one of them, by the way. His rate as improvement is abnormally high, which isn't only the case because he's abnormally talented, but also because only his plays after the unban got tracked, meaning that his hitcount is much lower than it should be.

Let's filter out all of these players and see what we get (small note: Filtering them doesn’t actually have an effect on the trend line, since our outliers are distributed almost evenly, but it’s done here for better visibility). Here's our new chart. Rohulk is still visible on it, way to the right:



looks different! We are now looking at ~9500 points of data, ~500 were filtered out

This looks very much like low correlation.

As we can see, the bulk of the players sits solidly between 0.0002 and 0.001 ... which is quite a large range! So we already know that different players have vastly different rates of improvement across the board, the fastest players improve up to 5x as fast as the slowest players. That seems like a lot, but it can be explained rather easily. Here are just some oft he reasons:

1. People are differently talented
2. some people don't play for improvement at all
3. some people are really good at farming and taking advantage of pp-maps wherever they can
4. some people just farm S-ranks instead of challenging themselves
5. some people get higher rates of improvements by playing offline
6. our variables aren’t perfectly adjusted for what we’re looking for
7. some players were only active in an era without pp-maps
8. it gets harder to improve as you get better. Low-ranked players tend to show higher rates if improvement than high-ranked players

Considering all of these factors, it makes sense that people would show vastly different rates of improvements. Let’s take a look at the trend line next.

The trend line points down, suggesting that retrying more actually means that you improve faster! So far the data seems to support CXu's theory more than Rohulk's, if we can even allow ourselves to say such a thing because of the very low correlation.

Didn’t I just say something about low-ranked players having a higher rate of improvement than high-ranked players? Maybe we should control for that? Alright, fair enough, done:



Player #9000 to Player #10.000

No matter what group you repeat that process with, the outcome is always the same. The rate of improvement always, ALWAYS goes down as people retry less.

There are many more things we could play around with to try controlling for different things (in fact I already did, and I’m willing to discuss the results and further ideas in this thread). Take this list of effects that could mess with our data one way or another:

1. We couldn't separate HR from DT players
2. Players with lower average play length could show faster rates because they "try harder" at the game, while people with higher average play length play more for enjoyment (listening to the whole song, playing for the music..), people with high average play lengths don't farm as much
3. Maybe the "never retry"-mantra is actually valid, but it only shows up for very high average play lengths (>400), where we have too few points of data to conduct a meaningful analysis
4. "Never retry" works, but it's such a long-term strategy that most people never get to benefit because they quit before it starts paying off
5. We still have no way to measure how much "variety" someone plays

As you can see, it's a complicated issue. But I want to wrap this up. This is what a large effect size looks like, one of the tenets that actually deserve to be on the list:



"Play more", large effect size, higher correlation.


In conclusion: We can’t really know if Rohulk is correct, but with all the data here I'm more inclined to believe CXu's perspective. Blindly following the theory that „sounds best“, is bad regardless. Whether or not Rohulk is correct (and he might be, as there are plenty of things we can’t control for with our data), I wouldn’t expect the effect size to be big, i.e. whether or not you retry a lot probably doesn’t matter that much. Don't treat this thread as a knowckdown-argument, it's important to remember that there are still many things that could potentially offset our observed trend.


„Play more“ still holds true, and so does „challenge yourself“, so what is there left to say?

Don't adopt a playstyle that feels bad to you, hoping to reap benefits that might not actually exist. If you like retrying a lot, keep doing it.





Discuss



________________________________________________________________________________________________
this post is part of a series
The reason you can (probably) never become a pro at osu
The reason why pp (probably) doesn't ruin mapping
The reason why tablet is (probably) better than mouse

stats lul
balance both

Cxu had another answer aswell which I can't find but he was basically saying that it doesn't really matter whether you retry or not, because you end up playing the same amount anyway, which i believe to be true. If you add up a couple tv size songs it's a marathon.

Or something like that maybe im butchering it. Can't fully remember but maybe you'll find it. I agree with that whatever that was.


But it kinda sounds obvious that by retrying you'll get pp faster, since that's just how it works, i mean if you shitmiss and don't retry and then play an ar 8 sliderfuck map and only fc the map you were trying 4 days later it would seem like you're improving slower, even though in theory you probably would have been able to fc it given sufficent retries.

Also you might want to take into account that both rohulk and jesse are mentally retarded, based on my professional analysis.

So i guess do whatever you want is a solid statement.

But i don't think anyone thinks it's really bad to retry other than the edgelord princesses, but then you have people who say dont play nf, dont touch mods, don't play short maps, don't play anything more recent than 2013, not to mention etc and so forth.
Which is probably also a bad advice.

You can't generalize stuff like that.

Obviously people who retry a lot on pp maps are going to "improve" (their pp) a lot faster than people who almost never do so and F2 all the time.

oh you're the guy who made that pointless elitism rant thread, no wonder

Rayne wrote:

You can't generalize stuff like that.

Obviously people who retry a lot on pp maps are going to "improve" (their pp) a lot faster than people who almost never do so and F2 all the time.

Farming boosts you short-term, that's obvious. The question is if it hurts you long-term or not. We can check that by looking at players that are already playing a long time (higher ranked players, generally)



The top 3k are already playing for a long time, so even if they had an advantage through farming early on, they surely would have lost that by now and instead started hurting themselves through their bad habits?

Not really. As I said before, it doesn't show up. If it exists, its probably a really weak effect.

Good god the detail is strong with this one.
Make a tl;dr pls.

Hitcount dosn't mean nothing.
Player who plays longer maps have more hitcount. And that dosn't mean they retry less.

In my personal case i can't retry without getting worst.

Pawnables wrote:

Good god the detail is strong with this one.
Make a tl;dr pls.

gladly

It has been claimed by many that retrying a lot hurts your improvement.

After carefully looking at some data, I concluded that this claim is unfounded. The data I used are the top10k players, I plotted (pp/hitcount) against (hitcount/playcount) to approximate a rate of improvement and a retry-rate, then checked if improvement increases or decreases the more people retry.
Turns out it doesn't. More details in full version.

ManuelOsuPlayer wrote:

Hitcount dosn't mean nothing.
Player who plays longer maps have more hitcount. And that dosn't mean they retry less.

In my personal case i can't retry without getting worst.

(hitcount/playcount) describes hits per play. It doesn't perfectly align with retry-rate because it is influenced by the players map-choices, I'm aware of that.
That doesn't mean that it's a useless variable, though. As I said before, while not being a perfect measure for it, it still approximates retry-rate. As you can see, Rohulk has the highest value. Players like Rafis have low values.
It's not perfect, but it's also not useless, there is clearly a positive correlation between hits per play and retry-rate.


Another nice thing about statistics: If you have inaccuracies, (like your values are always off by a certain margin in both directions), then that will not matter for the final analysis as long as have enough data to work with. According to probability theory, the inaccuracies will simply cancel out.

It's only problematic if your inaccuracies point in only one direction systematically, which shouldn't really be the case here. Not with the (hitcount/playcount)-variable, at least. The other one is far more difficult.

Retry less isn't bad advice.
I think players who retry more, force themselfs more than usually players what dosn't retry. Because you usually don't retry when you don't care a lot, or the map it's to easy to play what you do it perfectly fine, or hard enought to don't think about a fc be possible for you. So you don't push yourself.
Meanwhile retrying 200 times, you're searching for a good score each time. So you try to improve more than a non retry player.

In my opinion it's 100% a mental thing. Some players retrying, play worst and worst because they lose energy and focus as more they try at that moment, making them play worst and get bad habits.
Meanwhile other players with other mentality get more closer each try. And one try more means more chances to reach the goal, new knowledge, more improvment. Getting closer each try.

Maybe non retry players mentality put all the motivation and energy at first attemps. And retry players get more energy and more focused as more they keep trying on the map until get it.

2 years ago i was a retry player, searching for ranks like crazy. I was improving my plays. I had like 30-50 maps and i only was playing those over and over until get the fc. After long long time trying one day suddently i started improving. Sadly my pc broke and couldn't play anymore.

After a months break i back from play as a 50k player to be a 200k player in skill. Somehow i understand i was not enjoying the game, so i stop retry and start improving until now.
A few days back i started retrying on a goal to improve and trying to get higher ranks each day. I quick start getting really consistenly getting FC at 1-7 retry at some 5* maps.
After a week doing it i was totally unable to play 4* maps what i could consistenly fc at first try consistenly. My acc drop, my combo drop, my stamina drop, and became totally unable to fc maps what i did fc easy days ago.

Also starting get a problem what i never had before. If i get a miss i'm actually unable to hit any circle after the miss and instanly die at the map.

Seems like a big mental wall due to retry over and over.

There is something around there where you can retry and get way better than when you don't do it. But at same time If you retry you can lose skills.

Maybe one days you can retry all what you want and others you should not retry. Or you can retry all what you want but only at X maps, and if you do it at other maps, then you lose skills.

No idea.

The only thing what I'm 100% sure about this is retriying you lost stamina. If you want stamina don't retry. Can't get nothing more about my experience as a player who almost never retry.

Hi OP, I fixed your graphic for you. You can repost it now.



P.S.

You do realize that the people we give the "never retry" advice are a different subset of players than the top 10k? Do you really think the top 10k come to G&R for advice? More like 100k and up.

Using data from the elite playerbase to make conjectures about the bottom of the barrel dregs of circle clicking society? Peppy pls.

O, another cool post.

*grabs popcorn*

pandaBee wrote:

You do realize that the people we give the "never retry" advice are a different subset of players than the top 10k? Do you really think the top 10k come to G&R for advice? More like 100k and up.

how do I get better?

Just sayin

What if there’s so little people who actually retry less, that you’re just plotting graph of people who retry just as much as each other? What if the meta is mostly comprise of short maps and there’s not a significant difference between people who’s farming from a person who is just F2ing maps which also landed on short maps?

Your data still have too much other variables affecting it like inactive players, non meta players, just the fact that no one’s improvement graph has a linear progression shows that your data could vary based on when you collected the data.

I have nothing intelligent to contribute to this discussion, retry less holds in my mind cause of the “play from start to finish” part

N0thingSpecial wrote:

Your data still have too much other variables affecting it like inactive players, non meta players, just the fact that no one’s improvement graph has a linear progression shows that your data could vary based on when you collected the data.

There's certainly a lot going on with the data, which is also why we have such a low correlation, but most things shouldn't mess with the trend line itself. For example.. there is no reason to believe that non-meta players, inactive players and so on have a higher or lower hit per play ratio, so these types would probably be distributed evenly over the x-axis and therefore cancel out. Rhey're going to be the same as the outliers that I removed in the "Filtered"-chart. They don't actually have an impact on the result. As I said, if the inaccuracies don't distort the trendline down or up in a systematic manner, there's no reason to be concerned about them.
We're not trying to have an accurate assessment of effect strength or anything, so we don't have to be that rigid about things.

N0thingSpecial wrote:

What if there’s so little people who actually retry less, that you’re just plotting graph of people who retry just as much as each other?

We're looking at quite a good range for "hits per play", from ~150 to ~300. I doubt that they're just all the same. People who retry less are going to sit closer to 300, people who retry more are going to sit closer to 150. Occasionally you'll get the guy who plays a lot of marathons but apart from that actually retries quite a lot and then still ends up with a higher number. But again, the correlation should still be there.

If you're farming, your hits per play are probably somewhere around 175. Unless someone hits F2 and lands on haitai all day, he's not gonna beat that figure.

pandaBee wrote:

Using data from the elite playerbase to make conjectures about the bottom of the barrel dregs of circle clicking society? Peppy pls.

sure, why not? "Play more" works for new players just like it works for pro players. Rohulk is preaching "never retry" to everyone indiscriminately, and religiously follows his own advice. "Challenge yourself and push your limits" has always been a cornerstone of improvement no matter where you go or what you do, it doesn't just apply to osu. Do you see any good reason why extrapolating the results wouldn't work?

Railey2 wrote:

sure, why not? "Play more" works for new players just like it works for pro players. Rohulk is preaching "never retry" to everyone indiscriminately, and religiously follows his own advice. "Challenge yourself and push your limits" has always been a cornerstone of improvement no matter where you go or what you do, it doesn't just apply to osu. Do you see any good reason why extrapolating the results wouldn't work?

There are plenty. A few off the top of my head:

players in the top 10k are for the most part well established with the various skillsets of Osu. Beginners and to an extent intermediates are not.

Beginners tend to fall into the trap of focusing on pp farming to inflate their egos and small PPs (am i good guiz? look how talented i am, look how much pp i have xdd) which leads to lopsided skillsets, bad habits, frustration, etc. High ranked players won't have a lot of these same issues since their skillsets are already well established (other than maybe the frustration).

High ranked players are well acquainted with many different styles of beatmaps and have accumulated a large volume of plays on a wide spread of maps. Beginners have not. Most beginners have a paltry amount of beatmaps and would benefit from expanding their pool.


That being said, playing more will usually always let you improve regardless of whether you're retrying or not. So of course you can improve while retrying, but there are good ways and bad ways, better ways and worse ways to go about doing things.

Of course I support the idea of retrying.
That is why it is part of the core of my gameplay guide which you can check out here.
And always remember: Retry smart, play hard!

Endaris wrote:

Of course I support the idea of retrying.
That is why it is part of the core of my gameplay guide which you can check out here.
And always remember: Retry smart, play hard!

Wow what a shameless plug :^)

Personally speaking I usually play a map 2-3 times before moving on when I'm practicing that is.

my 2 cents: both have their benefits and draw backs.

Retrying a lot works in the short term, fails in the long term. When you retry the same map over, you will squeeze the pp you want from it, but ignore skill other maps may require. It's more a brute force method if anything. You will be able to do certain maps pretty well, and others not too well.


Variety fails in the short term, but works in the long term. You are going to do horribly on every map at first, but slowly improve on all of them as time goes on. You won't see sudden big pp gains from this, and so it may look less rewarding. However, you will have the skill set needed to handle most things thrown at you, but to a not precise degree.

You did a decent job of discussing some of the limitations of your metrics only to handwave them all away saying "I don't believe the numbers lie."

The core issue being that pp is a very bad indicator of skill for the purpose of this argument. The argument isn't that never retrying is the best way to improve at farming pp. It's that excessive retrying leads to overfitting. People just have a tendency to parrot the overly exaggerated trivially incorrect version of the argument.

Your results primarily indicate that retrying/focusing on short pp rewarding maps will increase the rate that you improve on pp farm maps. No shit sherlock, that much should be obvious without any data mining required.

On the flip side, there is no way to measure a player's ability to play every sort of map (a skill typically beneficial in tournaments) and not just a narrow subset of them that provides the highest pp returns on investment.

I'm not saying you're wrong, just that your data is not sufficient to support the strong conclusion you are trying to make.

It’s a bit unfortunate that we don’t have better variables to get a more detailed and precise analysis.
It was a good read though, thx :p

I don't suppose OP is well versed in the scientific method. If he was, he would know that the data doesn't come close to supporting the hypothesis that retrying doesn't hinder improvement, and he should just get rid of any implications that it does before his intellectual credibility goes down the drain.

Props to all the effort put into this, but I still think not retrying is a better way to improve at the game. Remember this: PP does not measure all types of skills in osu!

My two cents in this matter: i think that retrying ability should be used smartly. For example retrying a map 100 times just to get an proper fc is not a smart play. To retry a map 5 times to get down a certain pattern would be a better use. But perhaps the best use case is just where you know you should have done better. The first time you don't do very well for example environmental reasons, and then your next play is, indeed, a better play than your last. That would be a good use case for a retry. But indeed, the majority of your plays should be from new maps that are a little above your comfort zone

I repeated the analysis with data extracted today.

One problem with defining the rate of improvement as pp/hitcount is that, as seen in the data, the expected amount of pp given a certain hitcount is not linear, but rather roughly a monotonically increasing quasi-convex function. This means that the pp/hitcount value tends to be higher when the player is higher ranked (which contradicts the hypothesis previously made that low-ranked players tend to show higher rates of pp/hitcount). The solution given by Railey2 is filtering to narrowed ranges of ranking (which works is the range is small enough to make the relationship between pp and hitcount roughly linear).

Instead, I define rate of improvement as the expected hitcount given the amount of pp of the player, divided by the hitcount of the player. E(hitcount|pp)/hitcount. This has the advantage of being dimensionless value that is invariant towards non-linear scaling transforms of the measurements of pp, the formula for rate of improvement can be applied to all ranges of pp rankings even if the relationship is not linear.

A problem is that it is dubious to consider obtaining pp the same as improving at the game. I'd rather call the variable "pp farming efficiency" instead of "rate of improvement".

For finding the expected hitcount given the amount of pp, I did an isotonic regression of the data, which is a non-parametric fit that only assumes that the relationship between the variables is non-decreasing.



Using that fitted curve to calculate rates of improvement, we obtain this graph:



With a linear fit with r = -0.116515. This shows there is a very slight negative linear correlation between the average length of each play of the player, and the efficiency the player obtains pp.