For my company, it's quite often where there's a person who signed the LO and did not turn up for work at the first day. we even need to call the person to ask for the absence reason.

It is better that you get back to them and ask whether they want their offer letter back or not as you already rejected them. If not, you are going to land yourself in hotsoup by adding a bad record to their database.