It is not unheard of to have minimal symptoms, even with larger ANs like yours (mine was similar). Upon diagnosis I had no hearing loss, and minimal symptoms (only some facial numbness). However symptoms can appear suddenly (I experienced sudden hearing loss a few months after being diagnosed), and I remember Adrienne's story who experienced onset of trigeminal pain, with a similar sized AN. Also keep in mind that ANs may grow in spurts, with no growth for a few months (or years), followed by a rapid growth spurt. There is some speculation that pregnancy hormones might help speed then up, but (although I believe this was true in my case) I haven't seen any scientific evidence. There is no evidence whatsoever that they may stop growing in postmenopausal women.
I had exactly the same thoughts (i.e. why do anything, when I would be probably worse off after treatment). But in retrospect, I think it is too risky to leave a 3 cm AN untreated, because one rapid spurt and you can end up with serious brainstem, or 4th ventricle compression problems. I did get away with radiation (at least for now), because I was at the cutoff limit. If you are unwilling to do surgery, I would say to have your backup plan ready (i.e. surgeon referrals, choice of surgeon etc), so that once needed you will be ready for surgery.
Marianna