Yes, you should pay back. Well, my company actually includes this situation in the code of conduct/ethics, any overpaid by the company should take the initiative to report to supervisor and return the exceed payment.

IMO, you should.