i would say it depends on the service you work for, the service area you cover and the shift you work. for example, you work a day shift for a private transport company in a reasobly safe area. vest isnt necessary. you work a day shift for a city rescue in a bad area, coin toss on this one. night shift in detroit on the city rescue, i would say HELL YES!!!
bottom line, its up to you and whether you feel safe. if the vest makes you feel better, hazing be damned, wear it. dont let them discourage you.
for the record, i work a private service, night shift, w/ a major metro city m/a contract. i wear a vest when the calls sound like they might need it( bad area, pd on scene, unkwn emerg call etc...)
be safe