Carbide is too brittle to use free hand, HSS dulls too rapidly in the abrasive board. (I once bought a box of carbide "re-sharps" - some of the smaller ones broke while pulling the foam packaging straight up!)
Part of the problem is that Cu is generally unpleasant to machine; it tends to pull the tool in. That same pulling can lift small pads/traces off the board if the drill is constrained. Center punching subjects the board to mechanical abuse that weakens the bond between the Cu & the board.
You're probably going to trash a few bits & some boards while finding the tools & techniques that fit your budget.
Roy