Dynamic loading of networked objects

I'm making a multi-player game, which needs to have, a LOT of networked objects, these are destructible walls.

Any ideas of how to make dynamic loading? I think I would attach a trigger and keep a global table for each out-of-sight object.

I'm not sure what to do, if you've already implemented something similar I would like to know

You shouldn't need to handle any object that is outside the view frustrum, as they will be automatically culled by the engine.

If these are dynamic objects being spawned and destroyed frequently, then look into object pooling.