When you go in to have your car refinanced you will be essentially taking out another auto loan.
And when you take out an auto loan or any other loan the lender will pull up your credit score which will put a hard inquiry on your credit report.
This hard inquiry will lower your credit score a little bit and that's why refinancing your auto loan will affect your credit.
Good news is though that after about a month the hard inquiry should drop off your credit report automatically and your credit score should go back up.