I would tend to agree if you're using a normal drill bit. However, in my experience, the cutting edges on a step drill are so sharp that it tends to pull the bit through the panel if the speed is too high. Before you know it, one lovely 14mm hole - not good!
Unless it's red of course - then all you need is a ballpeen hammer!!!