Feel guilty about saying no to my manager to help fill a shift.

Does anybody else experience this? I don't know why I feel guilty. Maybe it's because I do like the people I work with and I subconsciously feel like I'm letting them down or something. I know the store will survive without me, and I really want to spend my day off relaxing but I can't shake the guilt away.