Okay, so should the arguments about "who did what" in his world be a valid point?

I think it should be, because we want to see what a character is capable to do, and what s/he has done in the past. For instance, if a character has only fight huge monsters in his world, it would be, in theory, difficult for him to fight something half his size, because he's not used to such battles, and vice versa.