Yes baby copperhead snakes do have yellow tails or more commonly yellow tipped colored tails
The yellow colored tip of the baby copperhead snakes tail is a good way to tell whether it's a copperhead snake or not and the yellow tip of the copperheads tail is also used as a lure for frogs, lizards and other prey items that the copperhead snakes feed on.
There is another snake that also has the yellow tip on the tail which is the venomous eastern cottonmouth snake so those two are the only two types of snakes with the yellow tip.
If you see a snake that has the yellow tip on it's tail it's either an eastern cottonmouth snake or a copperhead snake.