Breathing is some cold air won't harm you or your lungs but if you breathe in too much cold air then your lungs could be affected because of the dry air entering your lungs.
I know sometimes when it's below freezing outside and I have to be outside working then if I don't cover my mouth up and breathe in the cold air then I start losing my breath and have a hard time breathing until I get inside a warm building or inside my vehicle to warm up.
So if it's really cold out it's best to wear a scarf or something else over your mouth to protect your lungs from the cold as the cold can really take your breath away if it's really frigid out.