Yes a baby that is 6 months of age or older and is already eating solid foods can have and eat mashed potatoes with milk or just plain mashed potatoes.
If the mashed potatoes are homemade make sure the mashed potatoes are fully mashed and there are no chunks of potatoes in them.
The chunks of potatoes that may be in the mashed potatoes could choke the baby so make sure the mashed potatoes are fully mashed.
You could mash the potatoes and then cook them and then if there are chunks of potatoes you can use a blender to puree the mashed potatoes to make sure no chunks are leftover.
I've fed my babies mashed potatoes with milk and even a little butter once they were eating solid foods and they loved it.
Always supervise your baby or toddler while they are eating foods.