While you're at it try something different.
Build a gateway in their base instead and if his probes try to kill you, just keep running circles.
If the going is good get 2-3 probes there to help get the 2 gateways up.
Then chrono boost your zealots out.

Just ate one in a game, got me quite confused on what to do for the first time haha.
Usually was cannons. Was not prepared to see gateways instead.