321jurgen wrote:

So this is maybe a bit far fetched but it would be awesome if you could make a program that runs in the background that downloads and updates all the maps. Maybe I should start programming something myself sometime :/ I only do web related stuff and it would be good to explore some different areas. If so in what language should I write it?

Keeping every map would be quite space consuming, and would also make the game run inefficiently and become a nightmare for reprocessing.

Zallius wrote:

321jurgen wrote:

So this is maybe a bit far fetched but it would be awesome if you could make a program that runs in the background that downloads and updates all the maps. Maybe I should start programming something myself sometime :/ I only do web related stuff and it would be good to explore some different areas. If so in what language should I write it?

Keeping every map would be quite space consuming, and would also make the game run inefficiently and become a nightmare for reprocessing.

That's what I would've thought at first but it will only take like 100gb (I'm guessing) which isn't that much to me and reprocessing (25k beatmaps) only takes like 2 minutes for me and doesn't happen that often.

Oh goodness, it's been a while. I've just updated the code. One major thing i left out of the code is to set a timeout limit. Personally, i've had the program just sitting at one song or checking one page for hours. Not sure if anyone else has noticed this, because there weren't any messages about this, but it's fixed.

One other thing I've realized, I've used an RSS notification thing for Chrome for some time and have noticed that (either because of the RSS thing or bloodcat) the updates were kinda scattered. It seemed that it wasn't sorted at all and i may have missed an update or new song because of this. So i have gone back on my word and made it check 100 at a time. One problem i can see with this is, it'll want to download 80 songs if you haven't check for 100 songs before. I may add an option if the change between the number of songs last checked and just checked are greater than 20 or somethin like that. Maybe just add the first 100 on the list to start off and keep updating from there? Let me know what yall think.

I've also changed how i save the recently checked items. Instead of using just the IDs, i now just use the entire JSON item array. When comparing with JSON items, if the times for updates are different, then there should have been an update to the song and therefor needs to redownload. And in doing so, I've changed the file used from Recent.num to Recent.json. The program will check to see if you have the .json file first and update, if not it'll check for the .num file and update. In changing the file type, it will delete the Recent.num file and replace it with the .json file.

The last thing i added, and many people probably won't even bother with looking at it, are comments in the code. Basically anything to the right of a # is a comment.



Now for comments. For some reason, i became unsubscribed from my own thread. So i never got notified when yall commented here. Not sure why, but i'm resubscribing again so this shouldn't happen again.

Sir_Sandalot wrote:

RavenMac, you should make it in java. 1+ is a good idea
Also if it sorted all the ranked from the unranked in 2 folders and kept that up to date would be an awesome addition.

I'm workin on it. And (much more likely than with Python) maybe a GUI.

321jurgen wrote:

So this is maybe a bit far fetched but it would be awesome if you could make a program that runs in the background that downloads and updates all the maps. Maybe I should start programming something myself sometime :/ I only do web related stuff and it would be good to explore some different areas. If so in what language should I write it?

Language is all personal preference and what you know. As for doing this, each song added to osu is given an ID. You could see about gettin all of the numbers in a list/array and maybe checking the date created on your computer and comparing it with the last update on osu or bloodcat. I know osu you'll need to get permission to get the API for it, or somethin like that.

Sir_Sandalot wrote:

Python would be a good place to start but java would make it a lot more easier for other people to use it.

I had the same idea as you.

This is the reason why i'd like to make this in Java. That way nobody would have to download anything for it...That is unless they don't have java...

Zallius wrote:

Keeping every map would be quite space consuming, and would also make the game run inefficiently and become a nightmare for reprocessing.

Space consuming? Yes. Game running inefficiently / Reprocessing nightmare? Eh... it depends. Having a ton of songs, i haven't seen much problem playing the game. Preprocessing though, that takes a few minutes though.

321jurgen wrote:

That's what I would've thought at first but it will only take like 100gb (I'm guessing) which isn't that much to me and reprocessing (25k beatmaps) only takes like 2 minutes for me and doesn't happen that often.

I've got a folder of just the .osz files and then the songs folder. My songs folder is over 116GB with 16K song folders and almost 50K maps. Reprocessing takes about 2-3 minutes. It does add up, but i'm a very patient person. And i have 6TB of storage and am looking at upgrading fairly soon. So it's nothing.

Just updated the code a little bit more. I'm not sure how much detail i should go into what i've changed, so i'll just cover basics. I changed the song info display while downloading. It now shows what number of however many songs you are downloading (downloading 2 of 5). If there's an error, it won't display all of the info again. Just the id artist and title plus the error message.

I changed two functions and added another. One to cut down coding a little. The other was the song info function. Just a slight change. The added function is used to return the song id artist and title to a few parts of the program. It also can return the unicode info if needed.

I've added a little more information in the error messages. It gives a message as to what function the code had an error and then the regular error message.

Not a huge update, but i was mainly hoping to get the display for how many songs are being downloaded finished. Not entirely sure if there will be another update for python, unless asked for.

Im getting an error: http://puu.sh/bbsFo/8765d8d3ee.png
it immediatly closes the windows after opening the file.

Syrasu wrote:

Im getting an error: http://puu.sh/bbsFo/8765d8d3ee.png
it immediatly closes the windows after opening the file.

Sorry for the late reply. I was attending a convention.
For some reason when editing the code on Mediafire, it converts the first tab into 4 spaces.
I have to reuploaded the code in order to prevent this.

I'm getting this error for some reason.
http://puu.sh/cb1DE.png

EDIT
Saving the beatmap without unicode characters fixed the problem for me.

Aistify wrote:

I'm getting this error for some reason.
http://puu.sh/cb1DE.png

EDIT
Saving the beatmap without unicode characters fixed the problem for me.

I know exactly what the problem is. I've noticed this on a few songs before. When it grabs the title and artist, sometimes there are unicode characters in them. I've updated the code.

And for a side note. I'm no longer using mediafire to share the code. I'm using pastebin now. I hope this will help when making small edits like i have been doing. For some reason it saved my last edit as one entire line. With some programming languages that's fine, but not with python.

Is it even possible for it to check all the songs you have and download the ones you don't?

Aistify wrote:

Is it even possible for it to check all the songs you have and download the ones you don't?

With this code, no.
However, it can be done.

I've got some relatives coming in today, so I'm gonna be busy for a little while.
I can look at doing this afterwords.

Hangs on the first song, with or without unicode

thisischrys wrote:

Hangs on the first song, with or without unicode

I've just downloaded and tested downloading 4 songs with each for 2.7 and 3.4

2.7:

3.4:

Make sure you're using the 2.7 code with Python 2.7 or 3.4 with 3.4.
There are slight syntax differences between the two and some components are named differently.

Also without a screen shot or description of what's going on or what the error is, i can't help all too much.

Used unicode again and got that again. I did change the code to look through 20 and only that. Works with no unicode though. Also is it possible to add a progress bar (or just the %) with the download speed somewhere? (Sorry if it looks like I'm asking too much but I'm just suggesting additional features that might be useful in the future )

Aistify wrote:

Used unicode again and got that again. I did change the code to look through 20 and only that. Works with no unicode though. Also is it possible to add a progress bar (or just the %) with the download speed somewhere? (Sorry if it looks like I'm asking too much but I'm just suggesting additional features that might be useful in the future )

I may have time tonight to look at the problem.

I don't mind the requests at all. I'll look more into it. It may help me when i start the GUI for Java. But I've gotta get school work finished first.

I've fixed the problem. I wasn't using a function incorrectly. I'll look into the progress bar soon and download speeds soon.

I was trying to make a GUI and I came across a nice application to do so Glade 3. Why not try using that? I'm actually working on one myself at the moment.

I've been looking at the GUI builder in NetBeans. I'm currently working on a semester long group project for my class and learned how to use it properly. I already have a program running just like this one in Java, but I've been busy with so many things.

I will take a look at this though. It looks quite interesting. Thanks

RavenMac wrote:

I've been looking at the GUI builder in NetBeans. I'm currently working on a semester long group project for my class and learned how to use it properly. I already have a program running just like this one in Java, but I've been busy with so many things.

I will take a look at this though. It looks quite interesting. Thanks

;o Good luck with your school work

If you've already got working code in python, I suggest that you take a look at pyqt or pyside.

felicitousname wrote:

If you've already got working code in python, I suggest that you take a look at pyqt or pyside.

I know getting it to work in a GUI isn't hard, but i want to make it so that nobody will need to download anything before using it.
Virtually everybody has Java and that's why i'm trying to work more toward it.
Python is great, but people need to get the right stuff for it to run. Unless there's something i'm missing...

RavenMac wrote:

Python is great, but people need to get the right stuff for it to run. Unless there's something i'm missing...

You ARE missing something. http://www.py2exe.org/

I got an error http://0vn.eu/7k4rr.jpg

Ovnize wrote:

I god an error http://0vn.eu/7k4rr.jpg

Aww man. Not another indent error... I swear I've fixed this a couple times now.

Are you using the most current version? I've had problems with MediaFire in the past that caused this issue. I've switched over to pastebin now.
I assume you're using the correct version of python for the code or else it would have given a different error.

Besides those two, I should look into what Howl said earlier and just make it into an exe and just submit the source code.

I'm using the most current version of both your script and python ^,^`
And yeah, I agree with that

Hey, I wanted to ask if there was some way to get the program to only download ranked maps?

Lemmy57 wrote:

Hey, I wanted to ask if there was some way to get the program to only download ranked maps?

At the moment, there isn't. I'm planning on making the GUI version probably some time in December and that will have several options such as downloading ranked or approved maps.

I've gotta finish my college project before playing with anything else. This thing is worth 70% of my grade so it's just a little more important.

RavenMac wrote:

Lemmy57 wrote:

Hey, I wanted to ask if there was some way to get the program to only download ranked maps?

At the moment, there isn't. I'm planning on making the GUI version probably some time in December and that will have several options such as downloading ranked or approved maps.

I've gotta finish my college project before playing with anything else. This thing is worth 70% of my grade so it's just a little more important.

Ok, no problem! I can wait until then. After all this is just clicking some circles to pass the time.

Done with classes! I'm fairly certain i passed, but can't be sure until grades are posted.
I've learned quite a bit with NetBeans over the past couple weeks so i'm certain i can get this working.
I have just a plain command window java version working right now. Works almost exactly like the python version.
However i can add options to a GUI program much easier than command window versions.

Here's what i have as for the options:
Download location (where it's saving the files)
Song title format (ASCII or Unicode)
Illegal characters replaced with (unicode equivalent or underscore or maybe a custom one)
(＼／：＂＊？＜＞｜ or _)
Number of pages checked (10 songs per page; no upper limit)
Number of threads (simultaneous downloads; up to 4)
Status filter (ranked, approved, unranked, etc.)
Genre filter (video games, anime, rock, etc.; genre set for songs may not be accurate)
Mode filter (Standard, taiko, CTB, mania)
Search popular songs (I would have to add filters manually)

I remember seeing updating songs as one request. I'm fairly certain i can get this to work. Check download folder for already downloaded songs and search for their numbers or somethin like that.
One option that i've been thinking of is download from a certain date. Sometimes i don't have my downloader running after a reset or someting, so i check for the date the last song was created and start downloading from that date.

Let me know if there are any other options you'd like added.

Program worked fine a few days ago but now I only seem to get this error every time i start it up.

Sythini wrote:

Program worked fine a few days ago but now I only seem to get this error every time i start it up.

I've just tested both versions 2.7 ans 3.4. They are both working on my end.
What i think happened was that the site bloodcat.com was down for a little bit.
It happens from time to time. Go ahead and try again and let me know if it's working.

RavenMac wrote:

Sythini wrote:

Program worked fine a few days ago but now I only seem to get this error every time i start it up.

I've just tested both versions 2.7 ans 3.4. They are both working on my end.
What i think happened was that the site bloodcat.com was down for a little bit.
It happens from time to time. Go ahead and try again and let me know if it's working.

Thanks seems to work fine now

Any news about an upcoming update as I saw you talking about it a few posts back.

Sythini wrote:

Thanks seems to work fine now

Any news about an upcoming update as I saw you talking about it a few posts back.

From time to time bloodcat goes offline, so if there is another error such as this the program tries 5 times, then tries again every 5 minutes.

The new version, I've been preoccupied with other stuff; Not a lot of it is all that important.
I usually work on it when i'm in the mood, if that makes any sense. Programming is kinda boring, but can be challenging at times.
At the moment, i can get the program to work, but the GUI won't update as it does things.
I think i know how to fix this, but again doing other things.

And on top of all that, I'm looking for a job that isn't fast food.

hi i downloaded the program but this program download all map ranked not ranked ..... if i want to download only ranked map how i can ?

The program was designed to download every new map available on bloodcat.
There would be a bit more to add to the code to only check for ranked maps.

I know i was saying i was working on a java version.
I have no real reason why i haven't finished it.
I am thinking instead of doing it in C++ to get myself familiarized with the language again,
but with osu not having a limit on how often you can download maps this program is somewhat obsolete.

I would be better off requesting an API key from peppy and working straight from this site than using bloodcat.
I would probably never get an approved key if i kept using Python or if i did it in java, because needing the key
to work with the site needs to be in the code somewhere or the program needs to get it somehow.
At least from what i understand...

Nice program! Much appreciated

-Pake

Pakemania wrote:

Nice program! Much appreciated

-Pake

Thanks. It was a nice little thing to work on.

RavenMac wrote:

The program was designed to download every new map available on bloodcat.
There would be a bit more to add to the code to only check for ranked maps.

I know i was saying i was working on a java version.
I have no real reason why i haven't finished it.
I am thinking instead of doing it in C++ to get myself familiarized with the language again,
but with osu not having a limit on how often you can download maps this program is somewhat obsolete.

I would be better off requesting an API key from peppy and working straight from this site than using bloodcat.
I would probably never get an approved key if i kept using Python or if i did it in java, because needing the key
to work with the site needs to be in the code somewhere or the program needs to get it somehow.
At least from what i understand...

i have edited the program to fix for me the problem but i want to share because use this program is fantastic and with this edit i love completely the program
(the edit is a re-write of the link how this program download the song )
bloodcat.com/osu/?mod=json&p=/?q=&m=b&c=3&g=&d=&s=date&o=0/ this link download only the ranked/approved map it's easy to edit with other preferences

bloodcat.com/osu/?mod=json&p=/xxxxxxxx the x contain the preferences is generated after you have selected the preferences on bloodcate for example

bloodcat.com/osu/xxxxxxxx if you paste this x on the final parts of the link on the program you can download the song with preferences

sorry for the bad english


I hope you enjoy this guide if the creator does not want it I will remove the comment

Glad this is still helping people out.
I may have some free time soon to work on the Java / C++ version, and i had already planned on adding this feature.

RavenMac wrote:

Glad this is still helping people out.
I may have some free time soon to work on the Java / C++ version, and i had already planned on adding this feature.

I would like to help but to java and C ++ do not know anything that works on sites online because I'm a student and I am still studying yet these programming languages,probably next year I will be able to create programs like these

I got a PM a little bit ago about my program not working. This is because bloodcat had changed a few things with the json formatting and things went missing.

I have updated the program but the only draw back i noticed was there is no set amount to songs per page. I have been getting anywhere from 50 - 61 songs per page instead of the 20 like before. The program also stores the recent files differently and will need to replace the Recent.json file. If the program throws an error about the file being used, either rename the file to something else or delete it.

If any more problems come up, let me know.