Yes hydrogen peroxide is safe to use in your belly button and is a good way to clean the belly button and help get rid of belly button smells.
You can dip a cotton swab into some hydrogen peroxide and use the cotton swab to clean out your belly button and it will get rid of the bacteria and help keep your belly button clean and prevent it from smelling.
I use some hydrogen peroxide on a cotton swab at least once per week to clean out my belly button and after doing so I haven't had anymore smells from my belly button.
You can also clean your belly button using some apple cider vinegar or even regular vinegar which can help kill bacteria that can lead to a belly button yeast infection and white discharge.
If you notice you have white discharge from your belly button it can be caused by a yeast infection which the hydrogen peroxide or vinegar including apple cider vinegar can get rid of.