I'm more focus on the UI the phone is using. Because it is one of the counterpart that runs all your processes and if it's well optimized it'll improve performance slightly.

So, if a phone that has great UI but only has 3GB of RAM compare to heavily skinned and poorly optimized UI phone with 4GB RAM, the first one will win.