Yes putting insulated curtains on your windows can help insulate the windows and keep the cold air and cold drafts out.
I put insulated curtains over my drafty windows and noticed my house staying much warmer in the winter and heating bill went down.
Also look on the windows and see if there's any part of the windows that need caulking and caulk around those areas to help keep heat in and the cold air out.