I wouldn't worry too much about it. My nephew had his until he turned 5 years old. It comforted him so his parents saw it as no big deal.
He finally gave it up when he turned 5 years old.
But if you want to help your son give his up sooner. You can either take it away and deal with the crying and tantrums or you might try putting something safe but sour on the nipple.
You could also snip little pieces of the nipple off at a time until it finally has nothing left.