I don't think mine does it all the time. When I really pay attention and try to keep the drill motionless, I don't see it. However I do notice if there is say a stiffener behind a flange and I am countersinking the flange, I don't see it. I think it is the pilot drifting while I am countersinking, especially on thinner material.