forum

Yet another beatmap mirror + data API.

posted 2013-06-10T10:01:48+00:00

Total Posts

73

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-04T19:38:14+00:00

Hmm... Made small change in code as potential fix, how ever not sure if it will help.

Updated files:
Main EXE: http://miz.hexide.com/mirror/Beatmap%20Mirror.exe
Debug symbols: http://miz.hexide.com/mirror/Beatmap%20Mirror.pdb

P.S. There will be no search results if you will not enter at least one letter into any search field.

Piotrekol

Joined March 2010

Piotrekol 2013-07-04T19:46:15+00:00

working now.
(and found duplicate entry- http://puu.sh/3v6qW.png)

osuStats

[web] Extended pp listing

osu!StreamCompanion

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-05T05:00:56+00:00

Thanks for reporting, I could find actual bug in code. But what I found was that search was performed during beatmap update, since duplicate records are not in database. It should not contain duplicate result for any other similar searches, except for that exact one, since its cached.
It will clear up eventually

1. Yup, found few other duplicate beatmaps too. Will be fixed in few moments.
2. Fixed, those duplicate entries from search will go away once cache entry expires ( could take a while

)

Piotrekol

Joined March 2010

Piotrekol 2013-07-05T13:36:52+00:00

My friend can't run this program under 32Bit system(Win XP)

osuStats

[web] Extended pp listing

osu!StreamCompanion

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-05T13:52:43+00:00

Its .NET 4.5 app. It does not seem that it was released for WinXP.
So the only suggestion I could give now, is to upgrade.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-05T15:16:15+00:00

Added option to view beatmap details on this site:

If anyone has suggestions, ideas; feel free to share. If possible I will implement them.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-07T08:22:30+00:00

Added menu bar.
Moved settings to separate window.
Added ability to change concurrent download limit.
Added a bunch of links to 'Help' menu.

Recommended concurrent downloads are recommended to be kept bellow 6. Exception would be for people who have extremely good connection to beatmap mirror ( 8MB/s and up ).
Mine is set at 30 because I have direct 1Gbit connection to server.

And as always, updated files at:
http://miz.hexide.com/mirror/Beatmap%20Mirror.exe
http://miz.hexide.com/mirror/Beatmap%20Mirror.pdb

http://miz.hexide.com/dDld.png
http://miz.hexide.com/AbD9.png

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-17T17:56:04+00:00

Added option to disable question window when closing app.
Added download finished animations to download queue.
Some minor tweaks that I forgot.

Debug build: ( provides useful information in case of crashes )

http://miz.hexide.com/mirror/Beatmap%20Mirror.exe
http://miz.hexide.com/mirror/Beatmap%20Mirror.pdb

Release: ( does not display debug console, slightly higher performance )

http://miz.hexide.com/mirror/Beatmap%20 ... elease.exe

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-19T11:10:05+00:00

Spent good while configuring nginx and added some more feature to main api:

Raw access to beatmap file contents: /beatmaps/12191/content/raw /beatmaps/12191/content/raw/background.jpg
File hash calculations for raw beatmap files: /beatmaps/12191/content/raw/hash/noise/1.png
Raw file layout: /beatmaps/12191/content/raw/layout
Now all invalid requests should receive API friendly 400 response.
All responses are now cached for 24 hours.

Notes:

If requested beatmap was not extracted prior to accessing raw files, they will be extracted once and "Location" header will be served to request.
Documentations coming soon.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-20T09:48:04+00:00

It seems that there are allot of users that don't have user agent set. Who is writing apps/scripts/services that use this api, please specify user agent ( something relevant ) and if its public projects that has thread on this forum, please specify thread in Referrer header like "/t/137156".

Preferred minimal http request structure ( this is example ):

GET /search/maps.title.like.night HTTP/1.1
Host: api.osu.miz.hexide.com
User-Agent: Osu!Mirror
Referer: /t/137156
Connection: Keep-Alive

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-23T19:40:54+00:00

Added some basic stats: http://api.osu.miz.hexide.com/stats
ALLOT MORE coming soon.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-24T11:35:15+00:00

And I broke everything. Api responses no longer contain "status" => "....".
Response code 200 = everything is good. 300 >= error.

Documentations will be updated in a moment.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-07-24T11:46:17+00:00

Updated mirror app to support new api structure.

Debug build: ( provides useful information in case of crashes )

http://miz.hexide.com/mirror/Beatmap%20Mirror.exe
http://miz.hexide.com/mirror/Beatmap%20Mirror.pdb

Release: ( does not display debug console, slightly higher performance )

http://miz.hexide.com/mirror/Beatmap%20 ... elease.exe

Ephemeral

Inland Empire

Joined April 2009

Ephemeral 2013-08-19T16:11:29+00:00

this is an excellent, well documented service. thank you for hosting it!

Hexide

Joined January 2012

Topic Starter

Hexide 2013-08-28T08:43:47+00:00

Well looks like project is pretty stable state.
All features that I had in mind are implemented and documented.

I also wrote small php class a while ago that could be used with api:
Class: https://gist.github.com/Mizore/5939745
Usage: https://gist.github.com/Mizore/5939747

wwared

Joined October 2012

wwared 2013-09-05T16:03:53+00:00

A bug report (I'm not sure if you have another place for these:
When doing a GET /beatmaps (http://api.osu.miz.hexide.com/beatmaps), the documentation says the result should include:

		name: "31066 DJ YOSHITAKA - SEED",
		title: "DJ YOSHITAKA - SEED",
		type: "osz",

However, the API actually responds with, for all the maps:

        "name": "1 Kenji Ninuma - DISCO PRINCE",
        "title": null,
        "type": "Kenji Ninuma - DISCO PRINCE",

For some reason "title" is always null and "type" contains the beatmap title (and there is no way to get the type information).
It doesn't seem like this happens on other API requests, but I haven't checked them all.

Thanks a lot for this service; it's very well documented!

Hexide

Joined January 2012

Topic Starter

Hexide 2013-09-05T18:27:46+00:00

Whoops. Fixed. Issue was that i forgot to place comma in between 'title' and 'type' columns :v

It might still show broken version, should take up to hour for cache entry to expire.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-09-05T20:24:14+00:00

Also fixed minor big that caused all images returned from api to be full size.
Fixed issue where beatmaps with no wallpaper would still return status 200 response.
Now instead of 404 response on beatmap images, stock icon would be returned (also resized and cropped)

Hexide

Joined January 2012

Topic Starter

Hexide 2013-09-22T19:51:24+00:00

If anyone is curious in API source code: http://git.hexide.com/Miz/osuapi

Also threw in database dump ( no data ).
http://git.hexide.com/Miz/osuapi/commit ... 90cd51dbb6

Hexide

Joined January 2012

Topic Starter

Hexide 2013-11-22T09:01:19+00:00

Just realised that beatmaps were not updated for past 10 days (last successful update was on 2013-11-12 00:00:20).
Fixed issue and following beatmaps were updated:

Avail_old_1

Avail_old_1 2013-11-24T00:41:36+00:00

It's kind of broken

Download MP3s opens the queue window, puts the song in it, and closes it. (Both on debug and release.)
Download Beatmap crashes the app in debug, freezes the app in release.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-11-24T07:36:31+00:00

Avail wrote:
It's kind of broken
Download MP3s opens the queue window, puts the song in it, and closes it. (Both on debug and release.)
Download Beatmap crashes the app in debug, freezes the app in release.

Possibly due to missing directories where downloads pointed to. That is the only thing that comes to my mind at the moment.

You could try using latest version: http://miz.hexide.com/mirror/BeatmapMir ... 3adfd2.exe

Avail_old_1

Avail_old_1 2013-11-24T17:59:10+00:00

Everything is pointed to my downloads so I doubt it's a missing dir.
EDIT: Same thing happens with latest build..
EDIT2: Just noticed ( a couple hours late ) how the app actually downloaded the .mp3, well, a blank one.

Hexide

Joined January 2012

Topic Starter

Hexide 2013-11-25T12:38:45+00:00

Avail wrote:
Everything is pointed to my downloads so I doubt it's a missing dir.
EDIT: Same thing happens with latest build..
EDIT2: Just noticed ( a couple hours late ) how the app actually downloaded the .mp3, well, a blank one.

Does http://api.osu.miz.hexide.com/ open up properly for you?

Avail_old_1

Avail_old_1 2013-11-25T22:17:07+00:00

Hexide wrote:
Does http://api.osu.miz.hexide.com/ open up properly for you?

Yup. Works fine.

Howl

Joined June 2013

Howl 2013-12-22T18:35:41+00:00

?

It crashes every time I try to open it

Hexide

Joined January 2012

Topic Starter

Hexide 2013-12-22T19:16:12+00:00

Make sure you are using http://miz.hexide.com/mirror/BeatmapMir ... 3adfd2.exe and make sure you have http://www.microsoft.com/en-us/download ... x?id=30653 installed.

Howl

Joined June 2013

Howl 2013-12-22T22:37:13+00:00

After installing Framework 4.5 it works.
Thanks.

Hexide

Joined January 2012

Topic Starter

Hexide 2014-01-01T10:12:49+00:00

Small update:

Removed details window (because it was not about to get finished any time soon).
Added double click download.
Added option that would prevent/allow download queue window to be automatically hidden when download is finished.
Added bit more debug logging.
Updated address to project repository.

Release build: http://miz.hexide.com/mirror/BeatmapMirror.8077c5ce9.exe
Git: http://git.hexide.com/Miz/osumirror/compare/d903adfd2...8077c5ce9

Hexide

Joined January 2012

Topic Starter

Hexide 2014-01-25T09:48:55+00:00

Whoops, looks like last successful beatmap update was at 2013-12-26 00:00:11. I updated php and forgot to install one module, which led to beatmaps not updating properly.

All is fixed. Might take couple hours to grab all new beatmaps.

Lense_old_1

Joined May 2012

Lense_old_1 2014-01-26T21:50:58+00:00

I have a few things that could be improved:

5513 doesn't seem to be valid. When I download it I get an md5 of 57a75e4030beb6174089c6f91fcf9a5d while the info says 3fcfe7eae40a8fa86e2d151ee536735 (also md5 of the valid map from http://osu.ppy.sh/s/5513)
Edit: 19491, 8442, 14431, 17796, 41482, 17796, 18879, 12292, 13422, 1735 and 2533 all look invalid. Am I doing something wrong?

There are a number of spelling mistakes. If you want, I can make a list.

On http://api.osu.miz.hexide.com/doc#/get/beatmaps/id it says
"Request data: [0-9]+ Can be any integer, ranked beatmap id"
which is correct, but on every other page (e.g. http://puu.sh/6zh7e.png) it just says "beatmap id" which led me to use id instead of ranked_id everywhere.

Hexide

Joined January 2012

Topic Starter

Hexide 2014-01-27T07:07:01+00:00

Right, they don't seem to be entirely correct.
Ill fix it asap ( 3 - 4 days hopefully, since I'm in the middle of moving out )

Woddles

Joined December 2012

Woddles 2014-02-05T04:08:04+00:00

Hey, first of all I just wanted to say great job on the API. It's extremely helpful for me, having data request via MD5 hash.

One problem I ran into though, is when using the metadata request (Example: http://api.osu.miz.hexide.com/metadata/9e63d9a0b139c5953101e5eaf77af809), 'm_beatmap' and 'm_beatmapset' don't seem to be displaying (Image example).

Any chances this could be fixed? Would be extremely helpful so I only have to make a single request rather than multiple per map.

Thanks!

Hexide

Joined January 2012

Topic Starter

Hexide 2014-02-05T07:28:17+00:00

Woddles wrote:
Hey, first of all I just wanted to say great job on the API. It's extremely helpful for me, having data request via MD5 hash.

One problem I ran into though, is when using the metadata request (Example: http://api.osu.miz.hexide.com/metadata/9e63d9a0b139c5953101e5eaf77af809), 'm_beatmap' and 'm_beatmapset' don't seem to be displaying (Image example).

Any chances this could be fixed? Would be extremely helpful so I only have to make a single request rather than multiple per map.

Thanks!

Hey, that data is missing because it was not present in beatmap at the time of parsing. It will show up when will become available. Also multiple requests is not a problem.

Hexide

Joined January 2012

Topic Starter

Hexide 2014-02-06T15:31:45+00:00

Sorry for down time, ISP blew up.

Hexide

Joined January 2012

Topic Starter

Hexide 2014-03-17T20:02:12+00:00

Fixed issue where all output which was returned from cache did not have 'Access-Control-Allow-Origin'.
Now all responses have it ( its set to `*` ).

Hexide

Joined January 2012

Topic Starter

Hexide 2014-03-17T20:36:29+00:00

Just realized one feature was never documented or announced.
API supports AND OR logic for filtering, for example of supported queries:

where m_title like dark OR m_artist like m2u - Wide range of beatmaps, containing beatmaps containing only `dark` in title or `m2u` as artist
where m_title like dark AND m_artist like m2u - Only one result, beatmap which contains `dark` in title and `m2u` as artist

Also added into docs: http://api.osu.miz.hexide.com/doc#/get/search

Hexide

Joined January 2012

Topic Starter

Hexide 2014-04-17T19:28:57+00:00

Fixed issue with new php5 version which caused beatmaps not to be updated.
These (http://miz.hexide.com/IKgZ8MaU.html) beatmaps could have been out of date for up to 7 days.

- Marco -

Joined December 2011

- Marco - 2014-04-17T19:34:26+00:00

I can't click on tools :c

make osu! more awesome! | ♥️ | Need Help? | Lazer > Stable

Hexide

Joined January 2012

Topic Starter

Hexide 2014-04-17T19:35:25+00:00

marcostudios wrote:
I can't click on tools :c

Nothings there. Never actually anyone requested anything.

Please sign in to reply.

21 / 73

›

›

›

›