Yes using tap water from your sink is just fine for chinchillas to drink.
There's no need to use special water for a chinchilla although you can if you want too but the chinchilla will drink the tap water just fine.
Tap water will not harm chinchillas just like it won't harm humans or other animals.
Water you buy in stores is nothing but tap water that is filtered so if you want too you can filter out the chlorine etc in the tap water if you want too.
But if not you can just fill the chinchillas bowl up with tap water and the chinchilla will drink the tap water and be just fine.
My friend had a chinchilla and he gave it tap water all the time and it was perfectly fine.