I was wondering for a few days if someone designed an adaptive system to fit the subjectivity of difficulty, and here it is. I didn't read it entirely but it looks solid, wow. At the very least, you can hopefully dump this on a programmer and they will be able to do it without asking too many questions.
I need to read again more in-depth to be sure, but it also sounds quite heavy to compute all these things, especially to solve the differential popularity. I believe peppy would gladly try this in a hypothetical universe where he's bored and doesn't have a lot of features to add, bugs to fix, and real life to live, but if the complexity is not lower than quadratic just for adding a new player score, then your ideas are basically moot. In that case, you need to either find a way to make the calculations progressively without breaking the ranking too much, or hope that everything can be refreshed in one go periodically with a good initial estimate for the new submissions. But first, you need to implement a ladder with some dataset and make sure it's both accurate and fast, like someone else said before me. But where is the dataset ? I'm asking Google, he says fuck you.
There is still the osu!api to request a few things like you said, but the top plays are definitely not going to be enough. You can still request the score for a combination of map + player, but it will hit the request limit pretty quickly.
Anyway, I can't give a proper opinion without trying out first.
I haven't completely read the thread either (honestly I don't want to dive into the community too much), but I don't agree with some other propositions. After re-reading I realized I'm repeating some stuff that was already said, but I'm too lazy to remove it now :
@ B1rd
TL;DR :
Even if you don't like rankings, can't you just let us chirp and brag with each other ? It's not like we are affecting how you play, is it ? Hmmm.
Actually there IS a way to even affect solitary snowflakes like you (assuming you still do play offline) but it's at the end of my rant, sorry.
The long post if you want to bother reading monstrosities :
You want to judge without a number, using the community opinion. Well that's great, but this system has been tested for many kinds of competition for centuries, and it has always been clearly biased towards the popularity of a player. I mean, look at how you (and others) fanboy about Cookiezi. I'm not saying he's not the best, he probably is, and he might even have double the PP compared to the #2 with the system proposed in this thread for all I care, but a lot of people are clearly rating him higher than he should be, because lolhype it's Cookiezi ffs bruh. It's difficult for a newer player to become #1 even if he actually had better skills, because the gaming communities are biased towards old players. The same shit happens in Super Smash Bros about Isai because this game has a lot of subjectivity about skills even though the guy is tired (and retired) of the game and the fans are rating him on pure speculation. Bias can also exists (for better or worse) towards black people, overweight people, girls, ugly people, asians, gays, etc. For example I am biased against USA and would severely bring down any player from there just because I hate that 2nd amendment and because their "USA! USA! USA!" chant is fucking annoying. And hey you're forgetting about all the players that aren't Cookiezi or that-super-popular-guy. What's up with MY rank ? Nobody is going to bother, and it's really frustrating to not have a way to see how much I progress. This is also why scoring/speedrunning in video games needed the internet, because you couldn't see who was the best one and how to improve before you could slap your name on a list with a score/time/whatever. We need to compare each other or to show off at some point to make a hobby interesting, it's not simply a matter of "plz enjoy the game in a corner of your little world". Even when you work seriously on some project, you have to deliver it to the world someday, and people will judge it with whatever method is available. You can't just store it in a cardboard and forget it in your attic.
By the way, even if you judge with a single person who decides who goes where without comparing scores, you're still going to need numbers. Cookiezi is the best ? Guess what. He's at the top of the list in your head, "1" is a number, whether you like it or not, and the same goes for the players below.
On another topic, actually we DO have a judging team of humans doing work on osu. Look at the beatmap submission system, and how mappers like Sotarks are garnering attention because they denature the game, yet every kind of bullshit he comes up with automagically goes to ranked. I don't have a clue about his personality and never read a single post from him, but he is clearly mocking someone with
https://osu.ppy.sh/s/805224 (it's not like I can FC it but my opinion is still valid right ?). Look at that background, the diff title, the patterns. I'm actually afraid of clicking on the beatmap discussion to read QATs fawning all over him when they are actually the clowns of the story. I don't want to play a casualized game (giving out free PP isn't the right path to take) just because a small club of circle1-2jerkers decided to do memes instead of doing their job. Actually, maybe we should go the shaming route, and straight up balance the PPs according to how much a mapper is giving out to the playerbase. It's even funny how people talk about the mapping meta. The fuck they are talking about, there is nothing of the sort. You can only talk about a meta when there is a winning objective to reach against another strategy. Mapping the best PP/difficulty ratio isn't the objective of a rhythm game. They need to map the music/song correctly instead, and the only "opponent" they have to beat are the players. I'm not saying they should only map weird songs like an aspire, but hopefully you get my point.
I'm sorry for writing so much about the mapping and submission communities even though I hardly interact with them (people will probably say I don't know what I'm talking about because of that), but they still have a lot of effect on the rankings. If the system presented here can provide incentives to increase the quality of beatmaps and give them a proud reason to talk about a "mapping meta" without creating controversial dramas (because that's something humans are incredibly good at), then it's all the better. Still needs testing though.
@ winber1
Deep learning is a fancy tool. You said it was becoming reliable, but it's actually overblown by marketing shenanigans. It doesn't just work for everything, you're almost never going to hear about cases where it failed miserably (for obvious reasons, the successful applications are the ones who make the most buzz), and from a mathematical standpoint we can't prove an upper limit of the error rate.
Well if it "looks like" it works, okay, whatever. But keep in mind that it will only just "look like". You will never be sure. If it breaks down, you will not know why. This cannot be called science, really, but again we are calling "science" a lot of things that are mostly bullshit just because it makes people feel great, so I guess it's better than nothing.