You can bring the baby formula on the plane in reasonable amounts to feed your baby but you can usually not bring any bottled water onto the plane.
You can however ask a flight attendant for some water to make the babies formula and they should give you some water that is sealed up to use in making the baby formula.
Never use water though out of a tap in the plane for the formula because it's not very clean water.
If the airport security does allow a bottle of water through then it must be in a sealed bottle and usually be frozen solid before they will allow it through and the bottle of water must be full.