The 1/32 is probably moving more than the dewalt, that is a tiny bit, you should start with an 1/8″. Pushing on it by hand doesn’t mean much.
How far is your work material from your gantry?
How long are your axis?
How fast are you moving when the bit strays off course?