A gallon of unfrozen water weighs in around 8.32 lbs and when that same gallon of water is frozen it will weigh in around 7.9 lbs.
So yes water will weigh a bit less when it's frozen since when liquid becomes a solid it becomes less dense.
Water gets lighter when it's frozen.