If you dig too deep in your belly button you might scratch the skin or cause bleeding from your fingernails.
That cut or scrape in your belly button could lead to an infection which could make you sick so you should avoid digging too deep in your belly button.
If you do scrape the inside of your belly button and get some bleeding or a scrape then you should use some hydrogen peroxide to keep the infection from spreading.
If your belly button is just dirty then use a Q Tip with some hydrogen peroxide, vinegar or apple cider vinegar to help keep the belly button clean.
There's no need to actually dig very deep in the belly button as a good Q Tip cleaning will be good enough.
You won't rupture anything by digging too deep in the belly button but infections can occur so I would avoid digging too deep in the belly button.