Okay, I did some further testing and it turns out it really has to do with the objects being loaded.
I have a really empty skin that uses a lot of empty pictures (i.e. 1 empty pixel) and if I use it, everything is working fine.
It's the skin used in this video as reference: https://www.youtube.com/watch?v=h7EaYwkCetY
So it's probably something in the default osu! skin (since I'm using that one usually atm) that causes the lag. I'm not sure as to what it is.
I'm gonna try adopting my empty skin slowly into the default skin with more and more elements to see, if it's a specific file that causes lag.
Edit: Okay, as expected it's the 300 files (or their sparkly nature). As soon as I changed them to 1pixel files the game stopped lagging.
I have a really empty skin that uses a lot of empty pictures (i.e. 1 empty pixel) and if I use it, everything is working fine.
It's the skin used in this video as reference: https://www.youtube.com/watch?v=h7EaYwkCetY
So it's probably something in the default osu! skin (since I'm using that one usually atm) that causes the lag. I'm not sure as to what it is.
I'm gonna try adopting my empty skin slowly into the default skin with more and more elements to see, if it's a specific file that causes lag.
Edit: Okay, as expected it's the 300 files (or their sparkly nature). As soon as I changed them to 1pixel files the game stopped lagging.