Company flying us to the Caribbean but wants us to work the full week?

Company has a tax write off office in the carribean. They are sending us there for a “team building week” but are requiring us to work while we are there. Am I wrong for being upset?