Rosemary flowers are not poisonous to humans and in fact you can eat rosemary flowers since rosemary flowers are edible.
Rosemary flowers are most fragrant and flavorful just before the blooms appear on the rosemary flowers and the blossoms are best eaten when fresh and have a mild, delicate flavor to them.
Rosemary flowers are also sometimes used as medicine to treat some ailments.