That's how auto mode works. It will change accordingly based on the appropriate exposure to make footage look balanced.

You can lock the exposure value by holding/long press at your desired area, after lock you can point the camera elsewhere & exposure won't change.

Or you can try 3rd party camera apps that use manual settings, like Pro Mode on Samsung phones.