I can only say it is not the program sucks but you aren't good enough with the program to do that.
Select -> Colour range could be one of the ways.
And also content aware erasing.
If you don't need that specific person being used, there are many arch viz websites have freebies for you to download cut out people.