Usually around 12 weeks which is 4 months babies can begin regulating their body temperature and you don't have to do as much to keep the baby warm or cool after 12 weeks.
Babies can usually be a bit colder than adults before 12 weeks of age so if it's cold out you should put blanket sleepers on the baby for sleeping even if it's warm in your house because your baby may be a little colder than you until 12 weeks of age.
After 12 weeks of age the temperature of the babies body will be the same as your body.
So after 12 weeks of age if you're feeling warm then the baby will be feeling the same warmth and if you're feeling cold then the baby will also feel the same cold.