Do you actually hit him??? In my game, I don't even get a chance to move gems and they don't even attack when he passes their proximity.....