Just go to court and play the lawyer game la....no need to be afraid of him.... he is using this as a tool to make you bow to his demand... set a baseline for yourself... if he don't agree.. then said that you will pass your case to a lawyer.... he will definitely get less that what he is asking.

Even if really go court case, he is sueing, so he will need to provide all the facts and details... and I doubt he has... even if he saved all the details.. it can easily be counter.