Sounds like a ball joint issue to me. Did you change just the lower arm bushing or the whole lower arm? There are 2 bushings & 1 ball joint per lower arm. Another cause might be your tyres. Any uneven wears on your tyres? Are any of your tyres having different brands & threads?

I once replaced my tie rod ends during a routine alignment + balancing.... but the idiot worker didn't attach it correctly and it became loose.
So steering sometimes skewed, sometimes ok, sometimes not.

Luckily i found it out quickly. Otherwise would have gotten into a serious accident. If my tie rod came off, i would totally loss control, because the steering will no longer be attached to the wheels.

Since you crashed through potholes before, suspect might your lower arm problem. Most likely cause is:
1. Lower arm bend or ball joint loose
2. Tyre thread not same for left and right (since you checked with tyre shop, you can rule this out)
3. Check if any of your brake caliper is stuck. Does your car skew when braking too?

Other possible causes:
1. Wheel bearing is dragging
2. Broken engine mounting, causing torque steer as drive shaft will spin faster on 1 side
3. Drive shaft is tight on 1 side. Does your car knock when you turn the steering?
4. ABS system kacau. Go to workshop scan for error codes. Possibly wheel speed sensor broken too