Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. osu!
  3. Development

Osu Statistics and Advanced Profiles

posted
Total Posts
5
Dynam1cNET
24 posts
Joined June 2018
Topic Starter
Dynam1cNET

Hey There :>

So, i have that stupid idea of tracking profiles over a long period of time and then try to pull some nice Charts and Statistics from them. At the same time i want to build some advanced profile pages.

Why tho?

Just because i'm curious.
Currently i Have a small server in a datacenter that i dont use at all. So, i installed a Database, a small backend to track users with the v2 API and a domain that redirects you to a 404 at the moment.
I would say thats a Perfect start :)

Why do i post it here and tell a bunch of Randoms?

i want YOUR data too :) in the future after i figured some stuff out and have gathered some ideas of what could make a nice Advanced Profile i want to make it to a nice site for Nerds like me.

The basic idea as of now

  1. Build a Page that i can fill in with your data. The Project is long term so i want so see some sexy charts in like 1/2 years.
  2. Make the default site for the profiles osustats.me/userid/gamemode
  3. Let a user choose its own subdomain. in my case dynam1cnet.osustats.me
  4. Let the users download a SQL Dump of other users and their Statistics
  5. Pull some overall Statistics
  6. With the point above, provide a API that can give a really good Estimation of what PP value matches what Rank. (the more users of a wide rank spectrum i get the better is the result)
  7. Since i use the v2 API with the identify and public scope i want to give users the right to add other users to the track list. (i will be limiting the amount tho since i dont want to tank the API server lol) this should only be used for one or 2 friends that you are curios of or even a new player of wich you want to see the improvement rate.
  8. Upload Replays to get even more Data :D if you are like me and you have never deleted your osu folder then you have a bunch of replays in "../osu!\Data\r" for me thats around 30k replay files :D i just need to find a way to pull data from it.

Can you start Collecting my Data right now?

YES! and i beg you to add yourself to this Project. I actually hope i can work with a bigger Dataset and learn something new with this Project.
Just go to this link. You will get redirected to a 404 as of now but it worked.

I Have some Ideas

gimme gimme gimme :D
Just write a comment or text me. My discord DM's are always open too <3

Origin

like i said above, i started to track me and a few friends a year ago. here is our rank charts :D (the only thing i wanted back then)
The ring on the top right is the 300,100,50 count you get from the v1 api. i made it into a Profile grade :D

My Profile


Virex, he improved a lot i guess


So, thats how a Inactive player looks like :D



Update 17/05/22

So, i got a old Server from work and im planning to use it for this project. so, i now have 2 xeon e5-2660 v2's and 384gb of ram to work with. Also 36TB of HDD space. Someone could say its overkill and i will not come even close to use that but hey... what do i care right? :D. nah ofc i will use that server for other stuff too but i guess that should be able to handle pp calculations from replays. Speaking of wich, im currently researching all pp reworks so i can reconstruct the top 100 of players and a rough estimation of their ranks graph based on the data/r folder. So if you have any tips or want to send me your data/r folder for testing in advance hit me up.
One special case is a friend of mine Virex who actually never deleted his osu! folder. So in that data/r folder is his complete history. That will be so much fun to work with.
The data collection did work nearly flawless in the past weeks so im looking forward to it. i had a few hours loss due to a restart of the whole root server and me forgot to give execute permissions on one of the autostart scripts :D

Current Tasks

  1. Finding all pp reworks and build a calculator for each one
  2. gathering old data/r folders to test
  3. setting up the new server and set up a caching system, so if the database server goes down again i dont loose data anymore. (It annoys me that i didn't think of that before....)
  4. Create a backup server that can pull the data from profiles if the default one does not send a "im alive" ping.
  5. Create a tool for people to compress and upload said data/r folder
Last edited by Dynam1cNET , edited 1 time in total.
BluJellu
67 posts
Joined November 2018
BluJellu
Oh yes take all of my juicy data i love it when you take my data :)))
luxrayy
82 posts
Joined January 2021
luxrayy
fuck did i just get hacked
if i didn’t i wanna see how this turns out lol
abraker
Global Moderator
8,327 posts
Joined July 2014
abraker
Upload Replays to get even more Data :D if you are like me and you have never deleted your osu folder then you have a bunch of replays in "../osu!\Data\r" for me thats around 30k replay files :D i just need to find a way to pull data from it.
If this actually happens this will be a gamechanger for pp development. One of current issues is scarcity of replays to do analysis on. Idk how your servers will fare with needing to pull thousands of replays at a time tho.

Also you know this exists? https://ameobea.me/osutrack/
Dynam1cNET
24 posts
Joined June 2018
Topic Starter
Dynam1cNET

abraker wrote:

If this actually happens this will be a gamechanger for pp development. One of current issues is scarcity of replays to do analysis on.
I would actually never have guessed that replays would be a problem... :D But lets see if i can help with that.

abraker wrote:

Idk how your servers will fare with needing to pull thousands of replays at a time tho.
I see a big problem with the bandwidth. maybe i can distribute the whole backend to all of my servers. For spikes i can borrow some hardware and a few u's of space from work :D but to be fair, i expect some downtime on my end since i will make errors and its the first time i want to collect a shit ton of data. But on the long term i want everything to be smooth.

abraker wrote:

Also you know this exists? https://ameobea.me/osutrack/
yes, but it never fulfilled my need of just let it go, forget about it and check back in a year later :D
Also i really like the idea to let users download the raw data to play around with. i mean, i wish i had something like this back 2 years ago ^^

But for now everything is in baby shoes so i will try to move slowly but accurate. Since the backend for the data collection is up and everything gets backed up regularly i dont have the feel to rush.
Please sign in to reply.

New reply

0
1 / 5