Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. osu!
  3. Development

Reducing the amount of files

posted
Total Posts
16
Jalatiphra
294 posts
Joined April 2009
Topic Starter
Jalatiphra
Hi, iam just trying to copy my osu folder from an old quite slow hdd to a new one.
since its 40 gb and about 160k files it takes considerably longer then copying a single file.
You all know the problem, nothing can be done here.

but we can tackle the problem from a different side.

why not create a texture map of all the files when uploading a beatmap ?
implement a new section in the editor ? medialibary, where all the files can be dragged, specify a filename. internally generate a texturemap and only upload the texture map to the server. voila no more orgy of hundreds of thousands of jpg files.
this would also increase access times for osu.

i dont see a problem to make a backwards compatible converter for allready downloaded files, since all filenames and dimensions are fixed ? arent they?
Espionage724
876 posts
Joined August 2011
Espionage724
Hmm, as a slightly different suggestion, what if osu! had a "Backup" option that reverted all available beatmaps on your computer back to osz (or whatever formats might exist)?

This likely belongs in Feature Suggestions btw :)
Jalatiphra
294 posts
Joined April 2009
Topic Starter
Jalatiphra
then it would take even more time
first to pack it to an archive and then to copy the archive and then to unpack it.

thats not a solution, i could make a raw copy of the filesystem to achieve the same.

also your approach has to be done each time you wanna migrate.
my suggestion only converts once.
Soaprman
1,392 posts
Joined January 2008
Soaprman

Jalatiphra wrote:

since all filenames and dimensions are fixed ? arent they?
Not even close. Your Songs folder probably contains storyboard images in just about every size imaginable.

No matter how you divide it, it's still the same amount of data and you're still going to be limited by the read/write speed of the slower hard drive.
Jalatiphra
294 posts
Joined April 2009
Topic Starter
Jalatiphra
its a huge difference if i have to copy 150k files @40 gb or 15k files @ 40gb.

it took 4 hours . for 40gb , i copied the remaining 260 gb of games from the same hdd in about 2 hours


even with the file dimensions not fixed, ti wouldnt be that big of a problem.. just not that easy to implement
Hula
581 posts
Joined January 2013
Hula
SSD.


Problem solved.
Oinari-sama
osu! Alumni
3,403 posts
Joined July 2010
Oinari-sama
Most users don't migrate very often (once a year? 2 years?), so I don't see any urgency in implementing this.

Also wrong forum, post in Feature Request next time if you have a suggestion.
MillhioreF
Pro Tester
5,004 posts
Joined July 2011
MillhioreF
osz2 never actually extracts, so you get 1 file for 1 song. This is coming eventually anyhow.
Winshley
osu! Alumni
2,384 posts
Joined May 2011
Winshley

MillhioreF wrote:

osz2 never actually extracts, so you get 1 file for 1 song. This is coming eventually anyhow.
I was thinking about having a packed file for whole mapset. Now I know that this is planned.
Jalatiphra
294 posts
Joined April 2009
Topic Starter
Jalatiphra

Hula wrote:

SSD.


Problem solved.
i dont put games on a ssd which have no loading times at all usually
but yes, that would solve the problem.

and yes, there is no urge to implmenet this. i just wanted to discuss this issue since i was totally bored by staring at my copy progressbar..

and sorry for the wrong forum

osz2 would solve the problem as i see it :)
Winshley
osu! Alumni
2,384 posts
Joined May 2011
Winshley

Hula wrote:

SSD.


Problem solved.
If you have money, of course.

I even had hard time buying just a basic gaming mouse and gaming mousepad... :(
Mithos
636 posts
Joined February 2011
Mithos
for now, use WinRAR or Win-7 to make an archive file. This is one big file that you can transfer instead of the mass of little files. I suggest you don't delete anything before you confirm that the archive has transfered the songs correctly.
Ephemeral
Inland Empire
3,945 posts
Joined April 2009
Ephemeral
compress your files to an archive (7zip is good for large files) and copy it all at once
Jalatiphra
294 posts
Joined April 2009
Topic Starter
Jalatiphra
how dont you get it that making an archive of the files requires the same time reading the sinngle files as copying them directly? onnly that i nneed additional time to copy the archive to the other drive..

Example:

Osu -> Tranfer directly -> 4h
Osu-> To archive 4h -> copy archive -> 1h = 5h

archiving wont solve this - as long as the files arent archived allready by default from osu

or to make it even clearer:
direct copy : one read on slow drive . one write on fast drive

copy with archive : 1) make archiv : read from slow drive - write on slow drive -as iam making the archive on the same drive
2) copy archive : read from slow drive, write to fast drive
3) extract archive : read from fast drive, write to fast drive

if we make the archive on the fast drive directly ignore point 2)
its still more reads/writes of the full data set if you archive.
Winshley
osu! Alumni
2,384 posts
Joined May 2011
Winshley
Transfering a bunch of small files from NTFS to FAT drive is slower, and is even more noticeable if you're transfering into USB Flash Drives. Archiving it into a single large file makes a blazing speed differences.

If you're transfering files to similar drive format (NTFS to NTFS), then yes it's slower.
Agka
680 posts
Joined January 2009
Agka
OP doesn't know about osz2 packages that have been in the works (though I don't know its status) So you might want to know what's going on with that, as it sounds fairly similar to what you're asking.
Please sign in to reply.

New reply

0
1 / 16