Reasons to give 2 weeks notice?

I've received a job offer largely on the condition I start very soon. My current position is as a delivery driver at a company I'll likely never work for again. Is there any good reason not to quit the day before I start my new job?