My colleague did this. He has an old flip phone. Good battery life, but can't install anything to it. No clock in or clock out apps. No WhatsApp, no Slack, Skype or Teams.
He is the only one of the team who cannot be contacted out of hours unless you actually have his phone number, which he keeps secret as much of the time as possible.
And today I learned that he does have a smart phone, just work is unaware of its existence.