When a player (B) leaves the controlled player (A)'s registration then everything is working fine.
But if the controlled player (A) leaves the registration and Player B stays at the same spot (still sending it's position) then the state is not removed in unity because the server doesn't send remove state packages for deregistered partitions.