As far as I can tell, it only bites you if there's people you WANT to work with or it's a company you need to be able to go back to.
Often, these jobs will permanently view you as a terrible employee anyway because you'd leave at all. Like you're a traitor, their own personal Judas.
On a side note, if we worked for companies worth giving notice, we probably wouldn't be on this subreddit.