You have what is called social anxiety and you're not alone in this.
I suffer from social anxiety too and it's crippling. I always look out my window to make sure neighbors aren't outside before I go outside and if the neighbors are outside I stay inside and wait for them to leave or go back into their house.
If I have to leave I try to sneak to my car and leave without making eye contact with them.
I'm sure they think I'm some kind of psychopath or something but I don't care.