Nope, never tried punching someone hard in the face. I suppose the cheek and jaw bones would get in the way...but surely it can't be that bad?

Why not just use a big stick? Less damage to you...and if the end snaps off, then you've not only got two sticks to bop 'em with, but two sticks with jagged ends.