the whole thing can be solved in a simple manner.

1. read all the agreements or recall any terms your discussed if seller refused to sell due to any reason any penalty or etc... if yes just follow the terms - case close

2. if not... give a courtesy voice call (try avoid using text message) to buyer tell him/her ur reason and promise an immediate full refund of deposit paid .... proper explanation with apology....