This is normal behavior when flying in ”Stabilize”, ”Loiter” or other flight modes that are assisting you in leveling the vehicle. But, under a controlled environment, when testing or arming the motors for example, this should not be the case. If this is happening to you, you should Re-Calibrate your ESCs. Please check this video for instructions.