Weighing in here ..... As Kathleen said "everyone is different."
In my case, I have lost vestibular function on both sides. Following my first surgery, I was terrified to even think about driving when I could not walk a straight line. So I waited a full six weeks, of my own choosing, before my husband said, "Let's go to a big, empty parking lot and let you try it." I was amazed that I had no difficulty driving unless I turned my head too quickly to look over my shoulder to check lanes. It seems as though my balance difficulties are primarily related to when I am trying to hold myself upright. When driving, that is a non-issue, at least for me. I am still very careful to use all of my side and rear-view mirrors routinely, as well as turn my head less vigorously than before.
I do not have trouble driving at night, but rarely do so, because I am totally disoriented when walking in the dark, so I would just have to sit in the car wherever I was going.
Clarice