The reason people need to have permits to be able to build a house on their own land is due to safety concerns to make sure that the building codes are followed.
When you get a permit to build on your land the county or whoever does the inspections will come out and inspect the home before they issue an occupancy permit.
Once they issue the occupancy permit and pass the inspection of the home then you can move into the home and it's just to make sure the house is safe for people to live in.
They don't want the house to be a danger to someone if the builder built the house not up to code and something such as a wall etc failed and or a storm easily took out the house.
Getting a permit to build a house is pretty easy and not all that expensive compared to the cost of building the house itself.