Yes a mouse or mice can freeze to death eventually if they cannot find a good warm place to take shelter when it turns very cold.
During the winter months when it turns really cold the mice may seek shelter such as in a house or other warm building so that they do not freeze to death.
So you might notice you have more mice during the cold weather versus during the warm weather but you can still get mice in your home in both winter and spring and summer.
Mice usually do eventually find a place warm enough to take shelter so they usually do not freeze to death but if you trapped a mouse in a cage and set the mouse in a freezing environment then the mouse would likely die within a day or so of the freezing temperatures depending on how cold it is.