Unanchored instances and server performance
Table of Contents
"The fastest code is the code that never runs" — Robert Galanakis
For the longest time, SCI Pathos-III has had (relatively) bad performance on both the client and the server. This is due to a multitude of factors: outdated code, old assets that aren't optimised, and simply the fact that the map is sometimes too big to keep up with best practices.
So you can imagine the surprise I felt when I found out that the reason behind it was that 80% of our server's calculations were because of 13,075 unanchored parts - of which none should have been unanchored.
Why did this happen?
To be completely honest? We still don't know.
The strangest thing is that some instances of an asset were rigged correctly (fully anchored), while other instances of the exact same asset had unanchored children. My personal guesses:
- Human error: Maybe a developer forgot to re-anchor a part after testing physics.
- A plugin: Maybe a building tool unanchored parts unintentionally during placement.
- Black magic: Knowing the codebase of this game, I wouldn't even be surprised.
Fixing the issue
First, we had to figure out which models were behaving badly. The culprits were mainly alarm lights, elevators, doors, and a specific ceiling-mounted machine gun.
We debugged this by running a script in developer console console to visually highlight the unanchored parts and then logged which models had bad behaviour.
We then went into studio and proceeded to fix every single model (not individually of course, we batched them together). We also benchmarked the server performance and found that it improved by 80% after these changes.
I found it funny how, after all these years of desperately trying to find the issue, it was staring us in the face the whole time.
The great mystery of Pathos lag
On paper, our scripts look fine. Sure, some are outdated, but we use RunService connections responsibly and actively limit Heartbeat or PreRender usage.
And yet all this time, the issue was physics.
In Roblox, unanchored parts are governed by the physics engine. Even if an object looks like it is sitting still on the floor, the engine is constantly performing calculations like:
- Is gravity still acting on this object?
- Is it colliding with anything? (floors, players, etc.)
Now, imagine asking those questions thousands of times per frame.
Roblox servers run on a 60Hz tick, meaning the server checks the physics state 60 times per second. We can do some napkin math to see the impact:
784,500
Physics checks per second
(13,075 parts * 60 Hz)
Wow.
To put this into perspective, our next most physics-heavy system was 20,000 times less impactful than these unanchored parts.
Back to the quote at the start
As developers, we try (and sometimes fail) to optimise our code in the best way we can. But in this case, our fix didn't come from code, but from ensuring that the engine didn't have to run its code at all.
Turns out that the most powerful optimisation wasn't a complex algorithm or a clever solution - it was just giving the server fewer things to think about.