How to enable bullet hit decals on invisible objects?

I'm optimizing my game by having separate simplified networked collision models (invisible) from the more detailed non-networked client model (visible). Its working, and consuming less network object resources

However, when a player fires onto the invisible collision models you cannot see bullet damage details.

How would you solve this problem, and make the bullet damage decals visible?