Yes at 4 weeks of age puppies can drink water and are usually beginning to wean off their mothers milk so you can introduce water to puppies at 4 weeks of age.
Some puppies are even ready to begin drinking water as early as 3 weeks of age so once the puppies reach 3 weeks of age you can begin supplying water to the puppies by then as well.
The puppies should learn from their mother to go to the water bowl to drink and eventually the puppies will be weaned off the mothers milk and drink only water.
Just keep water out for the mother and for the puppies and they'll start drinking the water when they're ready to drink it.