You don't have to wash new clothes before wearing them if you don't want too but the main reason for washing new clothes before wearing them is because of the excess dye that can be in the clothes and transfer to skin.
Also another good reason to wash new clothes before wearing them especially if you bought the new clothes in the store is it's possible that someone has tried those clothing on in the stores dressing room so it could've come into contact with someone else's skin.
So you may not be getting very clean clothes even when you're buying them new.
I try to remember to wash new clothes before washing them but sometimes I forget and still wear the new clothes without washing them first and it has never been a problem for me.