Mount jittering when going backwards

In first or third person camera settings, if I set Movement Control Mode to Facing Relative, when going backwards my mount seems to align itself to some direction and keeps shaking left and right. Why is that? Is there a way around it (keeping this mode's behaviour)?

Edit: I've done a lot of debugging and it actually also happens when walking, though it's hard to see. For some reason, right after pressing S, the player is rotated left or right, the angle depending on the rotation rate setting. I've tried correcting that in a script but the function to rotate the player is server-only, and i can't send requests at client's update rate...