Using ReplacePlayerForConnection to switch between two GameObjects

I'm currently trying to use NetworkServer.ReplacePlayerForConnection() to switch between two gameobjects. One of these objects is the player spawned by the network manager, and another is a previously spawned gameobject. The first time I switch from the player to the other gameobject it works fine and switches, making isLocalPlayer false on the player object and setting isLocalPlayer to true on the gameobject. However, when I switch back to the player, it tells me the following message:...

Using ReplacePlayerForConnection to switch between two GameObjects