Cats are pretty robust and can tolerate some cold temperatures during the winter but when it gets below 32 F then cats can still be at risk of freezing to death or develop frostbite or hypothermia.
When the temperature gets below freezing cats in the wild will seek shelter to stay warm.
If you have a cat and it's stubborn about coming in the house and the outdoor temperature gets below freezing you need to force that cat inside the house even if it puts up a fight about it.
As for heat A cat can tolerate the heat better than they can the cold so they're much safer being outside in the heat than they are in the cold.
In the heat a cat can seek shade to stay cooler.
If you leave your cat at home during the summer months be sure to leave the ac on enough to keep the cat cool but not too cold.
As long as you keep your house below 90 F your cat should be just fine as long as you leave it plenty of water to drink too.