The only time I refuse something (usually I say, "ok, but I get it next time")...but when I am seriously like "NO, i've got it"...its because I feel like the guy is expecting more than just friends, and its my way of saying its not.
As far as looking for a girl in a museam or library, good luck! :/