You have to consider the costs incurred when you refinance eg. legal fees, valuation fees, stamp duties. This could add up to a bit.

Instead of refinance, why not call your banker and negotiate a "rate revision" as it would not incur those fees. Banks will only entertain those who ask.