The reason diapers are supposed to affect a boys reproductive health is that the diaper causes the groin area to stay warm and can keep the groin too warm which supposedly kills off the sperm.
Yes babies do have sperm as well as toddlers although it's not fully developed sperm like adults sperm.
However I wouldn't worry too much about it as many of us have worn diapers as babies and a lot of male adults today have worn disposable diapers as babies especially those plastic backed baby diapers of the old days.
I wore the old plain white disposable diapers that were made of poly backing which were much thicker than today's modern disposable diapers and so did my husband.
He was just fine wearing the disposable plastic backed thick diapers of years gone by and he was able to get me pregnant 3 times already.
So although the diapers can sometimes affect a babies reproductive health it's not really anything to worry about.
Also today's disposable baby diapers are made to be much more breathable than the old plastic backed diapers so there's less of a chance that the babies reproductive health will be compromised.