I faced this issue with my new TYR goggles.
First, you can try the fit using "vacuum test". Google this. If both sides stick well when tested independently, you may have same problem with me.

Turned out that the distance between the two sides was too wide,  even after using the smallest bridge. Solution: i used cable tie to connect the goggles and reduced the distance.

When shopping for new goggles, always do the vacuum test. Walk away if the shop does not allow testing.