A child with Chickenpox can still take a bath so go ahead and give the child a bath or allow the child to take a shower.
Bathing in warm water can also help relieve the itchy feeling of the Chickenpox and can help soothe the child's skin making him/her feel better.
A nice warm shower can also help with the Chickenpox or you can also place some Oatmeal in the bathtub which help with the Chickenpox too.