When you have a fever your body is rising in temperature to kill of the virus.
When your temperature is on the rise the outside of your body feels colder because it's really internal temperature being warmer and not the outside body temperature.
So you shiver and feel cold because your internal body temperature is warmer than the outside body temperature.
The chills are caused by your body trying to generate heat to warm itself by contracting and relaxing muscles and is why you can feel colder than other people in the same room with you.
Whenever I experience chills with the flu I will have an electric space heater pointed at me to keep me warm and have blankets.