Yes, it is an issue to balance when the visual cues for balance are reduced. A couple months ago I went to see a friend's band play, and afterward went stumbling out of the bar - not because I had had even one drink, but because it was dark outside!
I did a lot of work on it in vestibular rehab, especially because I work night shift as a nurse, and sometimes it's just not a good idea to turn all the bright lights on when you go into a patient's room, so we wanted to make sure I would be okay. Just like other balance issues, it seems like it's a matter of practice. We did the same exercises like Cheryl mentioned.
If you're looking for home exercises to help, I would try standing in a doorway or somewhere where the wall is close if you need it, and have someone dim / turn off the lights for you. First stand with feet at normal standing position, then together, then semi tandem, then tandem, then on one foot.. trying to master each before moving to the next and holding onto the wall when you need to. Throw in some head motions, like looking up and down, then side to side, with each foot stance and I bet after a few weeks of practice you'll notice a difference!