forum

The state of Bancho

posted
Total Posts
66
Topic Starter
peppy
I realise I haven't given much information about the reason for the recent downtimes during peak periods, so here you go. The issues are not hardware related, but rather the bancho implementation is being pushed to extremes. This stems from the sheer amount of data which is exchanged to provide every user with every other users' status, resulting in an exponential increase in server load with more players. I am working on a fix for this, and hope to have it implemented by next weekend. This does assume I can take the week off my day job to complete it (there's a lot to be done).

Until this is completed, you may notice some functionality temporarily disabled in order to keep bancho up and running. Currently this includes status updates (for in-game user panels) and rank letters being displayed on song select beatmap panels. Please understand these are temporary measures.

This graph should give you an idea of exactly how much more data we are sending when the player count increases from 3.2-3.5k up to 4k and beyond:


For those saying things like "peppy needs to upgrade the servers", please understand that I am aware of the capacity of the servers, and it it was resolvable by upgrading hardware I would definitely resort to this first.

Your patience is appreciated.
Dai
thanks for the info. I got many times DC today
we only have to wait
CapnDavid
I guess we have to wait for now.
Hiyorin
Great to know you are working on it
A week off work to fix this is dedication :)
Sakura
Nice to see that you're working on it and i sure have noticed that the ammount of players have increased so i kinda figured that was the reason anyways, good luck with this peppy and thanks
lepidopodus
Ah, so that's why. I was really concerned about recent lag, but well, at least now I know what is the problem briefly, so I feel relieved now.
_Angel
More and more people like osu ))
eddieee
gogogo peppy :D
Hika
Never really noticed much until now.
Thanks for working hard on osu!. c:
PandaCath
It's just proof that osu! is getting extremely popular :)
Verdisphena
Good luck
SoulAlchemist
Sounds like you are going to have your hands full with this peppy. Good luck.
AceproCX
Fight! :)
Backfire
Yay! Glad you're finally working on it ^^
Ever 14
poor bancho

suffering a lot
Topic Starter
peppy

Backfire wrote:

Yay! Glad you're finally working on it ^^
I don't even want to start to explain how wrong this sentence is.
Backfire

peppy wrote:

Backfire wrote:

Yay! Glad you're finally working on it ^^
I don't even want to start to explain how wrong this sentence is.
Oh wait, I see. Sorry D:
OzzyOzrock

Backfire wrote:

What do you mean...?
Finally working on it?
Dragvon
Good luck peppy, this is a sign from what's coming in a nearby future xD!
Let's get prepared for 10k players+ online :D!
Backfire
What I meant to say was glad you're working on bancho, since I was worried that it might go into downtime again, making me lose scores like it did last time. If that also makes no sense, then I apoligize.
Ayeen_old
Bancho having a rough week ~.~ . How about stop spectating others? Is that will help :?:
Topic Starter
peppy
a) Bancho going down does not make you lose scores, unless you were in a multiplayer game which is interrupted.
b) Apology is not required; it's always interesting to hear things from a users' perspective.

Stopping spectating won't really do much (I would disable it server-side if it did ;)).
Zekira
I'm assuming the red line in the chart is supposedly some sort of limit?
Topic Starter
peppy
The red line is nothing important. It is known as "95th percentile bandwidth calculations". The fact the red line is that high means this month is going to be expensive, though!
Ever 14

peppy wrote:

The red line is nothing important. It is known as "95th percentile bandwidth calculations". The fact the red line is that high means this month is going to be expensive, though!
lol expensive..
hope more osuers can become supporters
Ayeen_old
I got the money to donate, but dont have the methods to donate. I still saying no for credit card for now.
Rikers

peppy wrote:

The red line is nothing important. It is known as "95th percentile bandwidth calculations". The fact the red line is that high means this month is going to be expensive, though!
-Donates.....socks-
Is that enough to help :3?
mekadon_old

peppy wrote:

The red line is nothing important. It is known as "95th percentile bandwidth calculations". The fact the red line is that high means this month is going to be expensive, though!
Wait what. I hope more people donates so that peppy won't be bankrupt and we lose the game D:
Topic Starter
peppy
I never said it was a problem. Don't jump to conclusions please. Also, I wouldn't consider donating to Japan if it meant Japanese osu! players lost their form of entertainment (I think that would be an overall negative contribution).
DeScarlett
Wish you a good luck on doing your job 8-)
Aoitenshi
You are donating for Japan's crisis, For How long, I suppose?

I hope bancho still survives & osu! doesn't loses its appeal until you upgraded the harddrive/server/Idon'tknowexactly.
Topic Starter
peppy
Way to miss the point. Again. I *highly* suggest reading the original article before posting.
lepidopodus
peppy is right. Data is increasing by expotentially, so expanding server won't help cause it will increase server capacity arithmetically, so server capacity will be full soon even peppy expands server without fixing those issues. So that's why we need to fix fundamental problems first, I guess?
Topic Starter
peppy
Data isn't the problem, unfortunately (data can be easily fixed by allocating more bandwidth). The problem is in implementation.
lepidopodus
Well, that was the way I interprete the situation. I'm no expert about those things, so I guess I used those words without understanding them properly :/

Summary: I failed lol
Nadina120
8-) Always busy peppy 8-)
Darkimmortal
Good luck with the changes. I take it you're going with a complete architecture change (event-based model maybe?) or something along those lines?

Anyway sounds like an interesting problem - would love to help out :)
Xeno_old

peppy wrote:

Backfire wrote:

Yay! Glad you're finally working on it ^^
I don't even want to start to explain how wrong this sentence is.
Maybe you should take bancho offline for a week or two just to remind everyone you are doing this totally on your own personal time. Nobody is paying you to do this =p last I checked, osu! wasn't "Pay To Play". Now, let me run from the masses before I get slaughtered for suggesting something that can get me placed on a hit-list of some sort.

Anyhow, glad you are working on the problem. And I'm sure I'm not the only person that appreciates your hard-work. Keep up the good work. I'm soo gonna buy you a beer someday. (which might be soon, since I might be coming to Perth in a few months to give client based training on our software.)
anamorphism
glad to see you've identified the problem and have been working on possible fixes.

i'm wondering how much of the data that gets sent to us is really needed. the online users list comes to mind. i imagine it isn't a huge piece of the puzzle, but is that information sent to each user at regular intervals or is it only sent when the list is shown client-side?

the multiplayer scoreboard is also something that could probably be cut back in terms of how often the information is updated. is that data sent even if the player has scoreboards hidden? also sending each note hit is something that i don't find extraordinarily useful. it's often pretty distracting if multiple players are scoring near 100% on a beatmap and latency causes all of those players to swap positions constantly because of the gaps between where the user is at in the beatmap and the data that has been received. it might be more useful to take out the individual note score images and just send a full list update after every note grouping or something.

just thinking out loud, i'm guessing you've already thought about all of this, heh.
mm201
Changes are streamed. Yes, this is a major resource hog.
Topic Starter
peppy
User info is only sent when being displayed. Multiplayer scoreboards are not togglable yet. I won't stop you posting thoughts on implementation, but I don't need them either ;).
anamorphism
'tis my nature as a curious and arrogant software developer. my apologies.
beerBanKsY
its ok

be relaxed

take your time about it
[Sean]
How about... TWO BANCHOBOTS!!?!?? One for the Northern Hemisphere and one for the Sout-.... Wait a minute...

I'm just thinking, with Japan currently in crisis, we're still hitting 3.8k users. Imagine the load increase when Japan is back up and running. It may take about 2-4 months to see some sort of increase, but it's going to be big.

"LOLZ! BANCHO GO SLOW N CRASH? MORE RAMZ! KBYE."


I'd say an overhaul of what's really being deemed as "too much to process" sort of thing. What ever is doing the most damage in the form of load time, see if there's any way to lower the amount it's sending to help with it.

If not, just chuck in another strip of RAM. RAM fixes everything!

*[Sean] Has fled the scene*
Topic Starter
peppy
Let me guess: one male and one female bancho so they can have banchobabbies? :roll:

I think you misunderstood; the fact there is a crisis is increasing the load. It should drop down after Japan recovers it's infrastructure. (A lot of MMOs have been down in Japan to reduce power/network load, including some that share very large player bases with osu!)
PokemasterBrian
Right, peppy. We could see a decrease in load if Japan's repairs are done.
wilky_sandy
oh my!.. that's why i can't play OSU with my friends...
:o, i hope OSU will be good as fixed well :D
BTW thx 4 the information :cry:
IkkiMinami
thanks for the info. I got many times DC today
we only have to wait


_________________
This is the best signature ever :D
Topic Starter
peppy
Bancho is currently running at full capacity. Disconnections today/yesterday will be your end (or the routing in-between).
DJ Angel
full capacity? wow.
show more
Please sign in to reply.

New reply