May seem insulting but don’t take it that way, Use the prick punch then use a center drill, this will start the hole where you want it, then you are left up to how sharp and how properly the drill is sharpened to determine if the hole will drill straight thru …or not… also use of a good cutting oil will also aid in your endeavors.
I was taught, many years ago, when hand drilling holes in a drill press to prick punch, then spin the center drill backwards by hand to get it as close as possible before turning the machine on to have the center drill cut. Should get you within .005
http://www.richardsoutboardtools.com
classicomctools@gmail.com