Yeah you can always attend an interview with company B and see how it goes. My friend accepted an offer from Company X (haven't signed the contract yet) but before she started at Company X, company Y offered her a better position.

Another case my friend had two interviews with two competing firms and when he did tell them that he is attending two interviews with both the firms, the firms tried to outbid each other. Pretty nice and interesting if you ask me but that was in the UK.

As for rejecting, a simple call will do and give them a reason why you are rejecting them.