My awful manager has “invited ” me to a dinner and a pool party at her place.

She said she wants to go to a restaurant in town with me and a few ex-coworkers (I got them fired). I need out and idk how to deal with the situation without setting her off. I cant afford to quit but I can't fucking stand her. There is also a pool party but j plan on saying I got called into my other job. I want to be upfront about not wanting to go to the dinner so she doesn't ask again but idk how.