forum

osu!db v1.2: Read osu!/scores/collection DB's (Source inc.)

posted
Total Posts
19
Topic Starter
Primula_old
osu!db is something I've been working on for quite a while because my feature request for a accuracy sort method was rejected. ;--;
It's also going to be used in two of my other projects:
  • osu!np - Will allow for a better method of retrieving map data and also additionally output score data.
    osu!mp3 - Allows for assuming collections as mp3 playlists, so you can listen to songs from your collections (and additionally export/output them).
It's mainly intended for use by other developers to create projects from but anyone can use it, it features:
★ Ability to list collection & scores databases.
★ Notable columns of which can be ordered include the Accuracy% row of which the Osu! map list does not support.
★ Determines whether maps that you previously ranked on are installed and can direct you to the download page of the map if you click it. (Uninstalled maps will have a deep purple background)
★ The installed state of maps can also be sorted in the last column.



Note: The databases do not update in real-time, they only update when Osu! has closed so you'll only get results from the last time you closed Osu!
- There isn't much commenting, but the code is easy enough to understand. I wrote it in VB .Net (VS2012.. 2010 might work), converting to C# is easy enough if that's your preference.

WARNING: THIS MAY CORRUPT YOUR SONG DATABASE

(Probably Outdated, for developer use only)
Download osu!db v1.2 Here
Download Source Here

Change Log
v1.2 - 15.9.13
  • ★ Added some more columns to the scores listing and added additional columns to the collections listing.
    ★ Changed 'Installed' column to first row to avoid issues with collection db having lesser rows.
    ★ Alternated row colours to make it easier to read.
    ★ May have done a few other changes, can't remember because I fell asleep. ;--;
v1.1 - 15.9.13
  • ★ Fixed issue with osu!.db not reading correctly due to dependence on whether Username existed.
    ★ Added checking of whether maps in the score.db list are installed, maps which are not installed will have a purple background colour and clicking such maps will give you the option to open the map's download page so you can manually install them. (Implementation in osu!np will auto-install these maps)
peppy
Keep in mind the db format changes every few builds. This has the potential to corrupt your database, or destroy your PC. I will not support any issues that arise from the use of such applications.

@developer: please use SDK for such tasks (or request more sdk exposure if necessary) rather than reading raw files.
silmarilen
this looks really useful (especially being able to order by accuracy) but if it has the chance to corrupt stuff im not too sure about using it
puncia
Nice one, I hope it will be improved so it will not corrupt anything and will be used safely.

A little thing: just check if your "D:\Games\Osu" default path exists when you look for the osu! folder, I just have a DirectoryNotFoundException here :P
Kite
Uh.. pretty sure corruption of stuff wont happen since this is only a reader...
Topic Starter
Primula_old

peppy wrote:

This has the potential to corrupt your database, or destroy your PC.
osu!db only reads what it is able to, if a new version of the db file changes anything, osu!db will throw an exception when it starts reading data incorrectly. I've provided the source because there's a chance I won't be here to update osu!db when such a change to the db files may occur, osu!db isn't initially a tool for use by the general community, it's just an example of what data can be obtained.

Accuracy sorting and checking whether maps are installed are the only useful features for the community provided by osu!db and they are features of which osu! should have implemented but one of which has been rejected for whatever reason. My MP3 playing project is based on another feature request of which was rejected (playing songs on the menu screen from collections only).

From my experience, it's not possible for osu!db to cause corruption as there is nothing being wrote to the database files. (I've tested numerous times with incorrect formats as I made several mistakes when building the reader - not once did it cause corruption)

I'm not an expert, but from my understanding the only case for which 'corruption' could occur would be when osu!db is reading from the file while osu! is closing (osu! only seems to write to db files on close) - osu! will be unable to write new scores to the db file. This isn't actually corruption though, on the next startup, osu! will perform maintenance which will add the missing writes to the db file.

Could you please elaborate on how a database reader is able to cause corruption / destroy computers ?

Again, I'm not expert so I wouldn't really know - you didn't really provide enough information for which corruption could occur, but nonetheless, it would be delightful if the osu! SDK provided such data and I would be happy to switch over to using it when such data becomes available.

I haven't actually been exposed to the SDK yet, I'll make a request for data reading, thank you.
peppy
I haven't tested it, so I can't be sure. I just add a warning to all apps which are listed on these forums which open non-public osu! files, since I want people to be aware they are not checked by me. You're free to keep distributing this as long as you leave the warning, though.

p.s. you can already play songs from collections in osu!.
pielak213
­
sellyme
First up, the biggest issue is that sorting is broken. You're parsing the numbers as strings instead of converting them to integers when sorting, meaning that "900" is bigger than "5000", because 9 is bigger than 5. Additionally, you may want to define custom sorting for "1-Ranking" and "Mods" so that D and None respectively are last.

Secondly, it seems that osu! is the only mode that it picks up, so unless that's a bug, the "Mode" column is fairly redundant. Regardless, there should be the option to properly show/hide columns by right clicking on an existing column and selecting what you want to see, otherwise optimising the data you see is near-impossible; columns you've hidden by setting the width to 0 are very difficult to find again.

Apart from that it's great, but the sorting issue makes it pretty much unusable, unfortunately. Should be a 30 second fix, though.
Topic Starter
Primula_old

peppy wrote:

I haven't tested it, so I can't be sure. I just add a warning to all apps which are listed on these forums which open non-public osu! files, since I want people to be aware they are not checked by me. You're free to keep distributing this as long as you leave the warning, though.
This is the first time I've seen such a warning but no worries, not too bothered now that you've removed the word "WILL" from it, thank you.

peppy wrote:

p.s. you can already play songs from collections in osu!.
What are you referring to? I meant playing only songs from collections on the menu in osu!
As it is currently, there is no option to only have songs from individual collections play on the menu with auto advancing.

There is an option for playing a single song from a selected collection in the 'Jump To Window' but when the song advances after finishing, it switches to a random song that may not be within the same collection. This is bothersome when you wish to listen to only specific songs (eg. not marathon maps or maybe just rock songs if you have a collection of rock songs) from osu while doing other things on your computer.

pielak213- wrote:

osu!test crashed my database too many times for this to be effective accuracy fixer and i'm also not supposed to complain about it
Too vague to provide any solution to your problem.
osu!db only has access to the database files when you load them, so there shouldn't be any conflicts with it while osu!/osu!test is running and there's no need to reload them until the next time osu! is closed.

sellyme wrote:

---
The application provided is mainly just an example of the capable output, I'll update it when I get time to with better error handling/sorting but otherthan that I probably won't update much else until there's any big changes needed.

My implementation of it in osu!np will be more optimized, though that probably won't be until the SDK provides methods which meet peppy-standards.
peppy

Primula wrote:

There is an option for playing a single song from a selected collection in the 'Jump To Window' but when the song advances after finishing, it switches to a random song that may not be within the same collection. This is bothersome when you wish to listen to only specific songs (eg. not marathon maps or maybe just rock songs if you have a collection of rock songs) from osu while doing other things on your computer.
bug; please report :P.
RaneFire
Can you not get it to write the osu! file data to the application's own db file and only overwrite it upon a successful retrieval from the osu! file? Or does it do that anyway... What does the reader have to do to osu! while it's open?

This would help when osu! receives format updates, hopefully preventing the write-protected scenario as well... we would still have access to old information (somewhat useful) and could use it until the application receives an update.

If I misinterpreted anything it's because I don't program.

I backup all my data anyway so I'll give this a try.
Backup because... This Is Africa.
Topic Starter
Primula_old

RaneFire wrote:

Can you not get it to write the osu! file data to the application's own db file and only overwrite it upon a successful retrieval from the osu! file? Or does it do that anyway... What does the reader have to do to osu! while it's open?

This would help when osu! receives format updates, hopefully preventing the write-protected scenario as well... we would still have access to old information (somewhat useful) and could use it until the application receives an update.

If I misinterpreted anything it's because I don't program.

I backup all my data anyway so I'll give this a try.
Backup because... This Is Africa.
Yep this is possible, good suggestion.
The reader does nothing to osu! while it's open.

Will get something like this implemented with the other changes.
I'll also note that this is already possible to do manually in the meantime (change the directory to osu!db's directory and copy the database files to it).
pielak213
­
Miterosan
not working (index out of range error) reminds me of OutOfMemory :D
Piotrekol
there is a lot of new data(changed recently) in osu! database so that is probably expected.
Topic Starter
Primula_old
I don't intend to update this if it is out of date, will update first post to warn of such.
It's pretty much exclusively provided for other developers to learn from anyway.
Though hopefully the sdk gets updated (I did make requests a while ago) so that we don't need to delve into the db files anyway.
Tenderz
Hey, sorry for bumping this thread.
I was wondering if there still isn't a proper API for accessing the osu .db files (like the song database and collections).
If not, is there any documentation about the collections.db/osu.db format.

Also the source link is dead so I can't use that as a reference :o
Flex_unicorn
Hi all! I want to replicate different databases, and find out this thread. DB tool is not accessible for download.
Can you provide dl link, please?
Please sign in to reply.

New reply