I don't have social anxiety either, nor am I really an introvert. I'm not afraid of people, I'm afraid of being in places that aren't my home (I'm pretty textbook panic disorder with agoraphobia). Having people around exacerbates the fear because on top of that I have to add the fear of having a panic attack in front of people, disappointing people if I freak out and have to leave or not show up, and, especially at work, having people *depend* on me being somewhere other than my home. But I'm not *afraid* of the people.