Your credit score might not go up right away but you should see your credit score improve by at least a few points after paying off a loan.
It's a good idea though to keep a loan open as long as possible though and not pay it off early since keeping the loan open over the period of the loan shows that you're making payments on time and shows other credit card or loan companies that you're good at paying your loans back and they'll be more likely to loan you money or open a credit card account for you.
Paying a loan off early also shouldn't hurt your credit score but it does help improve your credit score by keeping the loan open for the full term of the loan.