Mold in walls takes around 24 to 48 hours to germinate and then can take another 2 to 4 days to start fully growing and spread to other parts of the wall.
Mold spores begin to colonize in 2 to 12 days and then it can take 18 to 21 days for the mold to become visible on the outer part of the wall.
So you could have mold growing inside the walls for awhile before even noticing it on the outside of the walls.
If the mold is on drywall you need to remove the moldy drywall and replace it with new drywall to get rid of the mold.