forum

osu!chatsearch

posted
Total Posts
4
Topic Starter
Howl
This project has been discontinued after being rewritten from scratch. The main reason is, it takes a huge amount of disk space due to all the logs being stored, and going back to the "delete every week" isn't an option. Although, if you've got a server, a lot of free space and are willing to set some things up, here's an open source web interface and chat logger built specifically for osu!:

https://github.com/TheHowl/osuchatsearch2
API docs: https://github.com/TheHowl/ocs2apidocs
non-working API example in C#: https://github.com/TheHowl/ocs2api-cs

Original post in its full glory
lately I had something to search in the osu!chat, because there was that topic where there was an interesting link that I didn't save, or I wanted to see the last message of that user. but searching through all the logs is a pain. So what's best other than creating a system that can search through the logs?

osu!chatsearch

Yes, I fast-coded (read: 3-4 hours of coding) a php script that fetches all the logs from corin's logs, puts them in a database to make them available for searching. shut up howl just give me the page where i can find it - here you go: http://howl.moe/ocs/ . You'll notice that the page is really lazy-done but, as I said, it was fast-coded so there wasn't any massive thinking about how I could design it. But, for the moment, it just works. so don't complain about the design and use it for what you need it, if you need to use it. Note that it works only for the past week

FAQ

why just one week?
resources. I have 250mb for the mysql database, and that's not enough. Consider that the logs on corin's IRC stats and logs take 66GB. That's damn a lot. And a month would take about 200MB, but still I have plenty of other projects in that database and chat messages per day will grow so only one week allowed for now, sorry.

open source please
not right now

Is there any generic search?
For the messages, there is the "%" wildcard. Just place it anywhere in the message form and it will work. It won't work in the user input box, tho.

why just 1000 results?
Pages will be added, but that 1000 limit is because otherwise the page would load slower and my database would swear in 3 different languages to fetch all the results.

why did you use the <pre> thing with the \t thing instead of table?
lazyness. it will change to tables, tho

Regexes

While the code is really early, I can't already release it as open source. So, what can I do is just give you the regexes that are used for the messages. feel free to use them for whatever you need!
function regexIt($msg) {
$re = "/\\[(\\d\\d:\\d\\d:\\d\\d)\\] <([a-zA-Z-_\\[\\]]*)> (.*)/";
if (!preg_match_all($re, $msg, $matches)) {
return false;
}
else {
return $matches;
}
}
function regexItAct($msg) {
$re = "/\\[(\\d\\d:\\d\\d:\\d\\d)\\] \\* ([a-zA-Z-_\\[\\]]*) (.*)/";
if (!preg_match_all($re, $msg, $matches)) {
return false;
}
else {
return $matches;
}
}
Explaining: the first function is for the normal messages, while the second is for the action messages. the regexes are done using the corin IRC logs formatting.
Topic Starter
Howl
Had some issues with the cronjob, so I had to do the thing manually. I mistakenly loaded the script twice, so there were dupes in the table. Should be fixed now, tho there might be some messages dropped.
Oinari-sama
Handy... Thanks for the app!
Topic Starter
Howl
Currently moving all the database and the website over to a new server, since it's using a new database I have to make this working again. It will hopefully return to work by tomorrow (I hope!). At least there's a good news in this: since I'm on a new mysql server, everything will be deleted from the database table every month, allowing you to search on messages up to one month old! (unfortunately, if it's the first of the month then you aren't in luck. I'll try to make it more flexible asap, unfortunately PHP doesn't seem to help much here. Or in general, PHP doesn't help. PHP is horrible).
Please sign in to reply.

New reply