Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. osu!
  3. Development

altOsu new way to download beatmaps

posted
Total Posts
60
show more
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Ok I've updated the crawlers, WIP and graveyard categories are being filled at this time.
It might take a while to get the all data of the 125 pages, after that, I think I will only do the first 10th pages for updates.

Sort is automatically updated to "last update" when WIP or / and graveyard are selected because there is no ranking date.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
User management is almost ready, but I'm not sure what Is important to show.
I think that remove / show beatmaps already downloaded is important, in order to do that, I have two possibilities;
- Create an applicatoin which browse local user folder songs
- Using osu api to get last played / scored songs

The first way is more accurate, but more intrusive, and the second way is less accurate (only the last 50 entries) bu does not require any kind of software to download.

What do you tihink ? what would you like to see after you entering your Osu! user id on the website ?
Last edited by Ikaer , edited 2 times in total.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
  1. links on beatmap cards (title, artist, creator, source, genre & language) can be used to create filter now.
  2. add source and tags in searchbox
  3. fix filter with url characters like &
otoed1
639 posts
Joined May 2014
otoed1
Oh snap this is kool.
Guy-kun
770 posts
Joined December 2009
Guy-kun
Hey, this is looking nice.
I'd be interested to possibly consider it for the download source for T-Aiko! (https://play.google.com/store/apps/details?id=guy.taiko) in the future if you kept development up.
A few critical things are still missing like paging and a few things still unintuitive (like filtering by one game mode requires 3 clicks to disable others) but it's promising.

(P.S. Downloads on Android also do not work due to some configuration issue, osu! site has this same problem but bloodcat doesn't, I think filenames have to be non-utf8?)

Would you consider a mobile UI?
Kibbleru
osu! Alumni
5,846 posts
Joined August 2013
Kibbleru
As for the user downloads or w/e,
I think you shiuld find a way to implement both.

For people trying it out at first, you may not want to download anything.
If the user likes it, then they can download the full thing
Tari
osu! Alumni
1,671 posts
Joined July 2012
Tari
It'll be cool if you could put a little bit more information in the beatmap description in each song tab.



From the above image as an example, Source should have a capital S and should have a distinctive change in font style to make it more noticeable when someone wants to look at source for the map. Also , you could put the word "Genre" on the left side to indicate that "Anime : Japanese means genre (even though its already obvious).

Also, I find it strange you labeled Mode, Difficulty, and Status on each of the Tabs and showed what they mean (like a legend) but somehow missed on or forgot to label the three buttons in the bottom of each of the song tab (http://puu.sh/jfG7f.png). In my honest opinion, you should label them :p
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

Guy-kun wrote:

Hey, this is looking nice.
I'd be interested to possibly consider it for the download source for T-Aiko! (https://play.google.com/store/apps/details?id=guy.taiko) in the future if you kept development up.
A few critical things are still missing like paging and a few things still unintuitive (like filtering by one game mode requires 3 clicks to disable others) but it's promising.

(P.S. Downloads on Android also do not work due to some configuration issue, osu! site has this same problem but bloodcat doesn't, I think filenames have to be non-utf8?)

Would you consider a mobile UI?

Hi, thanks :)

For now , the mobile version is not yet started. But yes, I think to make one, especially as the card size lends itself well .

I fixed the problem on downloading beatmaps on Android (missing some HTTP headers in downloading response, including as you thought, the content-dispostion which set the file name) . Tested and approved by your app :) (nice by the way: D)

For filters you're right, I will implement a new system ( first click selects a mode, the following clicks add modes , and start again once at 4).
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

Tari wrote:

It'll be cool if you could put a little bit more information in the beatmap description in each song tab.



From the above image as an example, Source should have a capital S and should have a distinctive change in font style to make it more noticeable when someone wants to look at source for the map. Also , you could put the word "Genre" on the left side to indicate that "Anime : Japanese means genre (even though its already obvious).

Also, I find it strange you labeled Mode, Difficulty, and Status on each of the Tabs and showed what they mean (like a legend) but somehow missed on or forgot to label the three buttons in the bottom of each of the song tab (http://puu.sh/jfG7f.png). In my honest opinion, you should label them :p
I will make some modifications in order to make each part more clear.
By label, you mean toolip or a popup like this: http://semantic-ui.com/modules/popup.html ?
Tari
osu! Alumni
1,671 posts
Joined July 2012
Tari
No, I mean http://puu.sh/jfHu5.png those labels on the top :>
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
@Guy-kun: I changed the behavior of difficulty and mode menus, hoping that they are more efficient.

@Tari: I added the labels to source, genre and language and use a different color for clickable links.
For the buttons "Play", "Download ", "Link", I can not put a label without losing a lot of space , cards are already big enough :) , but when mouse hovers a card , I added buttons borders, and hover them a pink border and then a tooltip after 1/2 second to explain functionality.
For additional information on beatmap (like tags, creation date, etc.) , I thought making the card clickable and then open a window with extra-information or something like that.


Probably, the next thing I will start is crawling user information when they are registered on website to add personal rank on card and played songs (/api/get_scores, /api/get_user_best and /api/get_user_recent from https://osu.ppy.sh/api/ should give pretty decent information to start with)
-Halycon
6 posts
Joined March 2015
-Halycon
A suggestion that will greatly improve this: Perhaps an integration with the osu! song folder a player has. So it will filter out the maps they have and haven't. Players with more than 11,000 beatmaps (like I do) will very much benefit from this. Since we don't want to download duplicates. But I do like the idea, since I'm a ctb player and wanted to just see CTB and osu! std maps.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

-Halycon wrote:

A suggestion that will greatly improve this: Perhaps an integration with the osu! song folder a player has. So it will filter out the maps they have and haven't. Players with more than 11,000 beatmaps (like I do) will very much benefit from this. Since we don't want to download duplicates. But I do like the idea, since I'm a ctb player and wanted to just see CTB and osu! std maps.
Yeah that exactly the point of the website.

To manage player song folder, like I said before, it would require a third party to know that information (because browser cannot acces to anything on your computer and that's cool for security). What I can do is crawled osu api to get player last 50 plays, but that's all, and in your case with 11, 000 beatmaps, I don't think that I will have the whole thing :).

P.S I know that for some songs of the website, the last ones, generally, download does not work, I work on this.
abalabahaha
18 posts
Joined April 2014
abalabahaha
Tip: add a DNS record (A or CNAME) for the root domain, altosu.org. Currently, only http://www.altosu.org works, http://altosu.org doesn't go anywhere. Not everybody is used to typing the www. in front of domains.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Oups, I didn't noticed that. Thanks for the tip @abalabahaha, should be ok now.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Empty download link shouldn't happen anymore. Let me know if you have one.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

Guy-kun wrote:

A few critical things are still missing like paging
Next page and previous page are now available.
Howl
1,121 posts
Joined June 2013
Howl
If you want to make a new post but you've already made one in the same thread recently, editing the previous post adding what you've got to say is a much preferred way than doubleposting, or in this case, tripleposting. Of course, this doesn't apply if you need to do a bump (in this case, the last reply in the thread was 8 hours ago, therefore the thread was already on top of the Development forum thread listing, and a bump wasn't required)
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
@Howl: Sorry about that. Next time, I will only update my original post for minor updates, but next one is a big one :)

Update 2015/15/08
This update is centered around the user management. You can create an account and attach your Osu! id (you can find it in url of you profile account).

Account creation requires email validation to prevent robots (the sender will be altosu.org@gmail.com).

When your account is created, the server will start to recover your data (last plays and high scores) from the Osu! api.
The api limits such requests to 50 entries. So at first, there will be little data. But gradually, as you play, the history will be updated and aggregated into my database. But you can try it easily by playing a song after creating your account, and then wait a few minutes, listing should be updated with the song you've just played.


When connected, you can
- Change and save your initial modes and difficulties (in user icon button -> profile). These filters are applied each time you refresh the page, but you can always apply other filters after.
- On beatmap listing, if you have played at least one difficulty of a song, the card will appear with a blue background and a rank should be associated with the difficulty you played.
- You can choose to only display beatmaps you have played or hide them from listing.
- The letter F can appear in place of the ranking, it is because you have played this difficulty but not yet completed it.

I'm not a fan of a blue background and Osu! rank standard icons, but well, it did the job.

Misc:
- added the possibilty to filter on song length. To do that, use filter icon to deploy top menu and enter values in seconds to filter beatmaps.
- Sort button have been moved to bottom and menu buttons have a little changed to prepare the small device version.

if something goes wrong, let me know!

P.S: As a new feature, it is possible (sure) that it will have bugs. If any of these bugs require modification / new structure of database schemas, I might be forced to do a cleaning. But I hoped that it will not happen.
Last edited by Ikaer , edited 1 time in total.
abraker
Global Moderator
8,327 posts
Joined July 2014
abraker
Oh wow, can search by difficulty. That functionality is broken here.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
2015/17/08 Update
- A new filter: File extensions to include in beatmap download. This is experimental, but you can if you want remove avi, wav, etc. from the download.
- Filters like duration, how to display played beatmaps and what type of file to download can be saved in your profile. You need to be logged in to access to the save buttons.

I think I forgot to mention it, but those extra-filters (duration, how to display played beatmaps, extension files to download) are accessible by clicking the filter icon.



dionzz99
677 posts
Joined April 2013
dionzz99
some ranked beatmaps aren't showed. any ranked maps before 20 june 2014 aren't listed.

another issue : difficulty filter seems messed up, some maps aren't showed (excluding the maps that aren't showed in the first issue up there)

X only
I and X
HIX
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Hi dionzz99,

For the ranked beatmaps which are not listed, I could use an example (a link to official website), it would be more easy for me to find out why they are not here.

For the difficulty, I think habits of official website have tricked you, when you filter on difficulties, let's say, Expert, list only returns beatmaps which have at least an expert mode, and when you download the beatmap, you only have this difficulty. It's like the Mode filter, when you had filter on Osu!Mania you only get this mode.

For example, if you take beatmap A, beatmap B, beatmap C (in that order) and only beatmap B has a difficulty expert, list will return beatmap B, if you add Hard difficulty and beatmap A has an hard mode, list will return beatmap A and beatmap B.

If you select all difficulties / modes, list should be exactly as official website (more or less one or two song, between 2 updates of the database).

Not sure I'm very clear :?
Rori Vidi Veni
175 posts
Joined January 2013
Rori Vidi Veni
This informative post will be deleted anyway, just move along
Last edited by Rori Vidi Veni , edited 1 time in total.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
I do not know anything about this game mode. Can you explain me what is a "keycount selector" ? or a link to a topic that explains.
dionzz99
677 posts
Joined April 2013
dionzz99
Oh my god...
im so sorry for this. I was just being stupid not seeing the "Next Page" button.

once again sorry ;_;
Rirakusu
218 posts
Joined November 2013
Rirakusu
Omg i love the site. Design + Functionality > Bloodcat + official site

However some problems with downloading maps. The link you provided to download a map (be it any map) does not show it's full size, causing problems with download managers and even Chrome's built in downloader. For an example, Warak - REANIMATE (https://osu.ppy.sh/s/186318)'s full size is 7.61MB, when I downloaded from altosu.org, Chrome brings me to 8MB and stops there without completing, 3rd party download managers (any that you can imagine) might even continue it all the way to 20+MB without stopping (I use Xunlei).

Why not direct links ;;
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
@jinhang_ang: Hi, first thanks :)

Then, for the downloading issues, I'm afraid, that I cannot do direct links. It would required that I prepare every possibilities of download on website server, which are countless :) (more or less difficutlies, mode, type of files, etc.).
When you click a download link, I unzip original osz files server side to get what you asked for and then I prepare a new archive (which might explain size difference with official website, compression is maybe not the same).
It's only at this point I can send in server response the real size of file, but it should be correct, and browser should know when to stop.
But It also can be an error that happened during the transfer.

I've not reproducing your issue with chrome built-in downloader and I don't know Xunlei :/. Did you apply filter on mode / difficulties / type of files before trying to download ? Can you post me here the wrong link (by using right click -> copy adress link on chrome) ? And if you are familiar with developer tools like Fiddler, you can use it to get the download response "Content-length" header and we will be able to compare yours and mine to see if something is different.
Poku
63 posts
Joined September 2013
Poku
Looks very nice and useful,The design is great, I love the way you built the website, good job :)
Rori Vidi Veni
175 posts
Joined January 2013
Rori Vidi Veni
This informative post will be deleted anyway, just move along
Last edited by Rori Vidi Veni , edited 1 time in total.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

Rori Vidi Veni wrote:

Ikaer wrote:

I do not know anything about this game mode. Can you explain me what is a "keycount selector" ? or a link to a topic that explains.
"Display only charts with X keys" option
keycount for mania is stored in the .osu file under CircleSize
Done, you can find the filter "Circle size / key count" in top filter menu next to duration.
I've also add HP drain rate, Overall difficulty and Approach rate filter.
Like usual, if you are logged in, you can save those filters to apply them each time.

Douae
25 posts
Joined February 2015
Douae
I really like this website, good job
Rori Vidi Veni
175 posts
Joined January 2013
Rori Vidi Veni
This informative post will be deleted anyway, just move along
Last edited by Rori Vidi Veni , edited 1 time in total.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
@PandaOG, Rori Vidi Venin, Douae; Thanks for your credits.

If you think about something else that would be usefull, don't hesitate, It will maybe not possible to do it, but if I can I will.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Hi,

New filters are available:
- Difficulty (based on number and not the categories Easy, Normal, Hard, Insane, Expert, cf. https://osu.ppy.sh/wiki/Ranking_Criteria)
- Play duration which is different from song duration (first note to last note without breaks).
- Playcount
- Played with success count
- Favourited count
- Negative user rating
- Positive user rating
- Approved date
- Last update date
- Submitted date

You can do nice combo with dates filter for example: Approved date > 2015 and sort by plays, it removes maps which are in top 20 for like forever (cf. https://osu.ppy.sh/p/beatmaplist?l=1&r= ... &o=&r=0&q=)

All filters can be saved if you're logged in as usual.
Sviundt
19 posts
Joined October 2014
Sviundt
Is it just me who have difficulties with logging into the site?
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
It's working fine for me.
Have you never be able to login or it is just now ? There is a reset password procedure if needed.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Filters & stuff are now in URL.
So now:
- so you can share them, for example, list of 2015 Osu!Mania beatmaps sorted by play count with only insane / expert difficulty and a key count of 7
- browser back / forward buttons are working
- F5 keep filters.
- to reset all filters, click on top Osu! icon.

+ minor fix: mp3 music sample now stop when you hit the button during the play.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Hi,

Last feature: a rss button (at top right corner).
Each feed is unique and based on current applied filter, so you can add multiple feed in your reader.
Feed title is a direct link to the download.
Feed description contains versions embedded in download.

Tested with feedly.

I think it's the last feature I will implement except if you want something else.
Arzenvald
4,056 posts
Joined July 2013
Arzenvald
@.@ nice, but i sorta confused with the way mapset sorted as 'ranked, pending, qualified, approved' at once..
i didn't notice those options are select & deselect able..

maybe you can add checklist symbol (or something else like color highlight) to mark whether its on selected or deselected, its barely noticable to see the difference between transparent icons (when i select it)..

>u<)b good job, neat layout too
zenithlight
49 posts
Joined May 2013
zenithlight
Is it possible to search based on BPM?
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

zenithlight wrote:

Is it possible to search based on BPM?
Done, available in filters menu.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
I've added a new option in top menu: the icon of padlock.
- when the padlock is locked, the filters are working as usual, you only get the beatmaps strictly matching them.
- when the padlock is opened, you get the full beatmapsets which have at least one beatmap matching filters.

Option can be save through > login > options
Iceclaw
2 posts
Joined February 2015
Iceclaw
[REMOVED]
Last edited by Iceclaw , edited 1 time in total.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Hi! It's because all filters you apply are reflected in URL in order to keep them applied if you hit F5 / refresh the page, and there is a lot of filters ! and also you can share those URLs.
I could simplifiy the url by only applying the "active" filters, but it's easier to handle in code this way.
ThePooN
osu!dev
371 posts
Joined February 2011
ThePooN
Looks great, well done! Is it open-source?

(en as-tu déjà parlé sur le forum français?)
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer

ThePooN wrote:

Looks great, well done! Is it open-source?

(en as-tu déjà parlé sur le forum français?)
Sources are available on https://github.com/Ikaer/osu-listing (for website) and https://github.com/Ikaer/osu-aggregator for crawler which get files and information.

(Et non, mais bon j'ai préféré en parler dans une section plus générale vu que le site n'est qu'en anglais).
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Hello everybody !

I wish you good Holiday Season and have plenty of PP and highscore under the tree!
And for this, I suggest the top 50 most played beatmaps approved in 2015.

Some minor changes in the website.
- a few design changes (in particular menus and the search that has been moved to the side).
- save uniformity (it is no longer separated between a part in Login> Profile and the rest in the filter menu). There is a unique save button for all filters applied.
- Added number of items to display per page (20-50-100).
- Added in save and in the url sorting and paging parameters.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
Hi,
New feature: youtube preview of beatmaps. cf screens belows.

How to:


If this thing is annoying you, you can disable it like this (option can be save in profile).


Note: To get a video, I search youtube with terms "OSU!+[Song]+[Artist]+[Version]", if nothing returns, i'm using "OSU!+[Song]+[Artist]". With this method, sometime, video will not match the song and / or difficulty of course, but it works for most of them.
Ikaer
46 posts
Joined June 2014
Topic Starter
Ikaer
A little up, website is doing well, but I'm still open to any suggestion to make it better :)
Royce551
66 posts
Joined December 2015
Royce551
Dang,this is a pretty good idea...
Good job on the website.looks pretty cool
Please sign in to reply.

New reply

0
10 / 60