Some types of phosphors are toxic and some types are not toxic.
Red phosphorus is not toxic in it's pure form and is a reddish violet powder.
Red phosphorus is used in production of methamphetamine.
Although red phosphorus can also contain yellow phosphorus as a contaminant that can be extremely toxic.
White phosphorus which is also known as yellow phosphorus is a high toxic waxy solid that glows when it's exposed to the air.
White phosphorus can be absorbed through your skin, inhaled or ingested and can cause organ damage, severe burns and even death.
The white phosphorus is often used in fireworks, fertilizer and munitions.
Glow in the dark phosphors which is often used in most modern glow in the dark items use non radioactive phosphors such as zinc sulfide and strontium aluminate, which are safe.
Although some items may use radioactive elements to glow continuously and these are also regulated to ensure your safety.