This is the way I understand it in layman's terms.
There is a reflex that keeps your eyes focused whenever your body moves (even the slightest move). That goes double when it's your head. It's part of the balance system. When you have AN surgery, the removal of one of the balance nerves disrupts that system. So instead of your eyes adjusting quickly and automatically when you move, it happens in a delayed fashion (a split second). I experience it whenever I move my head quickly to the left, but I'm fine when I move my head quickly to the right. When I'm walking quickly outside and lots of things are in motion around me, I sometimes feel slightly disoriented. I can see clearly, but something is "OFF" in the field of vision and I think it's related to this eye adjustment process. It's hard for me to tell how much of the disorientation is the fact that I'm still slightly dizzy when I walk around and how much is related to this other issue, but it's probably a little of both. You are almost certainly experiencing something along these lines when you have your head shaken. I'm not sure if this ever gets better. The first time I visited my surgeon he said it would, but the second time he was much less encouraging. I do some visual exercises to try to improve it (look earlier in this thread), but to be honest, it's the one area of my recovery where I have seen little or no improvement to date. It's not a terrible disadvantage though. I just feel that at times my visual perception is off.
Perhaps others that are further into their recovery can tell us if they experienced the same thing and whether it gets better.