Similar situations happened, signed bank offer letter, haven’t sign s&p yet. Agent told me to pay the legal fees n stamp duty if I cancel it. Is this true ?

Hi, my lawyer suggest me to talk directly to the s&p lawyer n I did so. He said will stop all the doc & pay them the processing fees which is about 600rm. And for loan side I already talk with the banker, will write a letter to waive off the penalty. The agent just trying to scare me.