Greetings,

Would recommend looking at the github README, has much of the same stuff as here but updated.

Background
So over the years our current tools have started getting a bit out of date, especially with all the recent updates to the Ranking Criteria. With none of them being open source, and since closed or abandoned, it's pretty difficult to just fix them or add things to them. As a result of this, I decided to make Mapset Verifier about 1 or 2 years ago.

Introduction
Mapset Verifier saves you from much of the tedious checking required for a beatmap to be ranked, making it a way less time consuming and unrealistic process. Many rules in the Ranking Criteria are straightforward, but with how many there are and how you just do the same thing on each map, this becomes super slow and gets repetitive real quick. Luckily, many issues are rare, so you'll often get away with just checking a few things in a more reasonable amount of time, but risk overlooking obvious things doing that.

This is solved by automatic checking, in the same way AiMod and Modding Assistant work. Of course, not everything can be checked this way. Things like metadata and timing are a bit too complicated to fully cover with a program, but about half of the Ranking Criteria can be covered this way, which is pretty good.



Goal
This aims not only to provide us with an up to date and open source modding tool, but also one which we can expand upon and use long-term. It supports check plugins which can be added to a folder to be loaded in the same way the default checks are, so if something is missing or I ever abandon this for some reason, the community can still continue adding things to it.

Features
Apart from being able to check stuff:

1. Automatic song folder detection
2. Automatic snapshots
3. Automatic updates
4. Integrated documentation
5. Changeable difficulty interpretation
6. Verbose mode (minor issues)
7. Plugin support (add check DLLs to %APPDATA%/Mapset Verifier Externals/checks)
8. Open source (see MapsetParser, MapsetChecks, etc)

Checks
All checks from Modding Assistant (apart from a few less useful ones) have been re-made for Mapset Verifier, fixing a few false positives and updating things in the process. You can find a full list of checks both in the application and on github. Currently (as of 2019-08-05), these are the checks included:

General

1. (Resources) Missing background.
2. (Resources) Too high or low background resolution.
3. (Resources) Multiple videos.
4. (Resources) Too high sprite resolution.
5. (Resources) Inconsistent video offset.
6. (Resources) Too high video resolution.
7. (Metadata) Additional markers in title.
8. (Metadata) Inconsistent metadata.
9. (Metadata) Incorrect marker format.
10. (Metadata) Incorrect marker spacing.
11. (Metadata) Unicode in romanized fields.
12. (Metadata) Incorrect format of (TV Size) / (Game Ver.) / (Short Ver.) in title.
13. (Files) Unused files.
14. (Files) Issues with updating or downloading.
15. (Files) 0-byte files.
16. (Audio) Audio channels in video.
17. (Audio) Too high or low audio bitrate.
18. (Audio) Delayed hit sounds.
19. (Audio) Incorrect hit sound format.
20. (Audio) Imbalanced hit sounds.
21. (Audio) Too short hit sounds.
22. (Audio) Multiple or missing audio files.

All Modes

1. (Timing) Two inherited or uninherited concurrent timing lines.
2. (Timing) First line toggles kiai or is inherited.
3. (Timing) Inconsistent uninherited lines, meter signatures or BPM.
4. (Timing) Unsnapped kiai.
5. (Timing) Inconsistent or unset preview time.
6. (Timing) Unsnapped hit objects.
7. (Timing) Unused uninherited lines.
8. (Timing) Wrongly or inconsistently snapped hit objects.
9. (Spread) Lowest difficulty too difficult for the given drain/play time(s).
10. (Settings) Abnormal difficulty settings.
11. (Settings) Inconsistent mapset id, countdown, epilepsy warning, etc.
12. (Settings) Slider tick rates not aligning with any common beat snap divisor.
13. (Hit Sounds) Low volume hit sounding.
14. (Events) Breaks only achievable through .osu editing.
15. (Compose) Abnormal amount of slider nodes.
16. (Compose) More than 20% unused audio at the end.
17. (Compose) Concurrent hit objects.
18. (Compose) Too short drain time.
19. (Compose) Invisible sliders.

Standard

1. (Timing) Hit object is slightly behind a line which would modify it.
2. (Spread) Objects close in time not overlapping.
3. (Spread) Multiple reverses on too short sliders.
4. (Spread) Too short sliders.
5. (Spread) Object too close or far away from previous.
6. (Spread) Too short spinner time or spinner recovery time.
7. (Spread) Perfect stacks too close in time.
8. (Settings) Default combo colours without forced skin.
9. (Settings) Too dark or bright combo colours or slider borders.
10. (Hit Sounds) Long periods without hit sounding.
11. (Events) Storyboarded hit sounds.
12. (Compose) Perfectly overlapping combination of tail, head or red anchors.
13. (Compose) Burai slider.
14. (Compose) Too short spinner.
15. (Compose) Obscured reverse arrows.
16. (Compose) Offscreen hit objects.

Taiko

1. (Timing) Hit object is slightly behind a line which would modify it.
2. (Events) Storyboarded hit sounds.

Catch

1. (Timing) Hit object is slightly behind a line which would modify it.
2. (Hit Sounds) Long periods without hit sounding.
3. (Events) Storyboarded hit sounds.

Mania

1. (Hit Sounds) Long periods without hit sounding.

Current taiko, catch, and mania checks are not specific to those modes, they just don't fit into All Modes due to one or more modes being excluded for the respective checks.

Examples of fixed false positives/negatives

Modding Assistant

1. counting extended break times as drain time.
2. failing to account for minimum SV (0.1x).
3. misinterpreting hit sounds on slider bodies as hit sounds on heads/tails.
4. completely ignoring storyboard variables and animation frames.

AiMod

1. failing to detect unsnaps when they are 2 ms too early.
2. not accounting for stacking.
3. using a vastly outdated star rating system, saying you need an easy/normal when you already have one.
4. using inaccurate playfield measurements to detect offscreen hit objects.

Download (.exe for windows 32/64-bit, tar.gz for linux 64-bit, will look into mac later)
^ Get the .exe file (other stuff is for auto updating) and run through setup.
Windows & anti-virus programs might freak out at the file being uncommon/new, but that's normal.

Bug Reports
Contact me on discord Naxess#7180 (if you don't have me you can always join the Aiess Project), if you find anything that definitely shouldn't happen, like there being red text over the whole screen saying that something went wrong or the program suddenly not working after clicking on a map. Give reproduction steps (what you did before stuff broke), link map involved if any.

Note

1. Always use your own judgment. False positives and negatives may exist (as with any modding tool), meaning some detections will happen even if they shouldn't and visa versa.
2. Game modes apart from standard do not have star ratings implemented and don't have specific checks either by default. The tool can still be used to check for general/all mode things like audio/metadata/files/unsnaps etc, though.
3. Do be careful about which check plugins you use, as they can be malicious. The plugins are executed by the backend of the application, so they can pretty much do anything the rest of the application can.

Thanks to TrippinBeats, Alphabet, Uberzolik, Nyxa, yaspo, and people from NAT for helping with testing & feedback / making sure my checks make sense / saving me from going insane when writing all the documentation.

YESsssssetyesysssss oooooh yeseessssssssss

Bless

poggers lads now i can yolo more

This is gonna be REALLY helpful to mappers, thanks for this tool!

god bless naxess

good, thank you

wow nice good job for making this thing owo)b

annndddd it checks audio i love you naxess

fuck yea more mapping tools!!

now i understand why Naxess is NAT with basically 0 submitted maps

MA IS GONE, GOD BLESS YOU NAXY ♥

perfection

Very interesting. Could you compile it for 32-bit version in future, please??

woah

My map widepeepooHappy

That's pretty epic ngl

cool

holy pog

owo

thanks owo
btw how to make it show on my desktop instead of just being in the resource manager ;(

Naxess is god...well, yes.

thank you