I’m inclined to say you sound slightly paranoid but I am generally at work during the day. I would echo what the posters above are saying – go about your business freely but be aware of what’s going on around you. Keep your eyes up (not in your phone or on the ground) when you’re walking around or waiting for the bus.
One specific thing that you should check out if you don’t us it already is NextBus. I know a lot of folks say it’s useless, but I take the 62/63 bus all the time and find NextBus to be fairly reliable. If you walk to the bus stop when you know there is a bus coming then you minimize the amount of time you’re standing around at the bus stop.