What if you got a dozen or so applications, and each recruiter calls your references? By the time you need the reference the most like you are about to have an offer, your reference will be so annoyed with people calling him that they won't be giving references anymore.
This can even make it impossible for you to get a job because your reference gets pissed off with so many recruiters calling him just to ask about you