Baby goats don't need to drink water as long as they're feeding off the bottle or off the mother goat and drinking the goats milk.
Once the baby goat stops drinking milk from it's mother or from the bottle then the baby goat will need to drink water.
Goats require on average of 1/2 gallon to 3 gallons of water per day and baby goats can be weaned from the milk after 30 days old and they should eventually start going for the water on their own but sometimes you may have to introduce your baby goat to the water to get them to take to it especially if you've been bottle feeding them.
You could replace the milk in the goat feeding bottle with some water instead to help introduce the goat to the water.