Big jump? No. Gradual increase as you pay down the loan? Yes. However, if you sell the car and pay off the loan you'll lose the points you gained while you were paying it down. Part of the Fico score is credit mix. Credit mix includes open installment accounts. When an installment account is paid off, it is no longer open so it isn't included in that mix. Thus the score drops. It seems silly that we get penalized for paying off an account, but that's just the way it works.
