Bathing a baby with cold water is not a good idea and it's not OK to bathe the baby in cold water as the babies skin is thinner than an adults skin and they can get too cold.
So you should use warm water to bathe your baby in to prevent the baby from developing hypothermia or getting uncomfortable.
The temperature of the bath water for babies should be just above 100 degrees F to prevent chilling or burning the baby.
If you do not have a bath thermometer, test the water with your elbow.
When you put your elbow in the water, it should feel warm, not hot.
Always keep a firm hold on your baby during the bath.
Always check the water temperature with your hand before bathing your baby.
Aim for bath water around 100 F (38 C). Be sure the room is comfortably warm, too.
A baby only needs a bath 3 times per week at most and you can wipe down the baby during the other times.
If you want to bathe your baby more often, that's fine, and if you only bathe your baby one or two days per week (but spot clean any other messes and keep their diaper area clean), that's fine too!