Electric capacitors can hold their charger a long time as in months to even a year without discharging on their own.
So even if the device hasn't been plugged into a power outlet or hooked up to electricity the old capacitor may still have a charge to it that could shock you.
I would never remove a capacitor without discharging it myself first as it can provide a shock that could harm or kill you.
You can discharge a capacitor by shorting the terminals of the capacitor with a screwdriver that has an insulated handle.
Touch both the terminals of the capacitor with the screwdriver and should spark which means that you've discharged the capacitor.