Sounds like you've hit something and knocked things out of alignment. On the safer side of things I would be under the front end right now inspecting components for damage or impending failure.

 

Check for play in all the three rod ends.
The steering rod end, the two suspension rod ends.
Its not easy with weight on them so you may need to go to a garage to get it checked.
I suspect its one end of the steering rod end. If its really bad it will cause this problem. But so will a loose wheel, or very bad suspension arms.

 

Awesome. Thank you for your help!
I had it into the garage today and they swung it up on the hoist and said the biggest problem they could see was the loose control arms (which I have coming).
I would have to agree with both of you with regards to the steering rods being an issue. Is there somewhere you would recommend I start? Something in particular I should be looking for? Is there a part of the rods that are more likely to be damaged than others?
Could it have something to do with the column itself? I figured it wouldn't as it feels fine the rest of the time but in that one location and unsure how the column could skip the wheel side ways.
Thanks again!

 

One thing to consider is that some of the joints seem to also have internal spring tension so its not always so easy to find the one that is loose. I have a dim memory of that on some joints so mentioning it just in case.