I understand what you feel like and I have to a certain degree the same anxiety that you have because I feel that I am intruding when taking pictures, but still I love it. Don't care, get some books from your favourite authors, read, learn, get their perspective and just go out and take pictures without giving a damn. Stand up for yourself and motivate yourself, set goals and pursue them with vigor and intent, and before you know it you'll forget about the so-called bad times.