With a score like that (spectacular! assuming it is a true FICO score), you should/will be offered the best rates. If you don't think you're getting the best rates (predatory lenders, brokers, etc.), keep asking questions and looking around for better rates. I think that the "here's what's bringing your score down" statement is misleading because your scores are so close to the top. When there's nothing really bringing your score down (because it's so high!), you end up with some bs excuses in that section of the report. So since you don't need to be worried about not getting the best rates, I don't think you need to be worried about how to "fix" your score.
Another piece of the puzzle is the mix of your credit (revolving (credit cards) vs. installment (student loans) vs. etc.). Getting rid of what sounds like your only "installment" loan may mean that you take a slight hit since lenders like to see that you handle different kinds of credit well. Depending on what else you may do with your money, and what kind of return you may make on it, it might be worth it to pay down that loan--certainly if you're paying interest over the life of the loan. But the "mix" is one thing you would want to consider before paying it off.
Once you start applying, ask your broker or loan officer or whatever, and ask them if it would make a difference to them. With scores like yours, I don't think it would make any (even incremental) difference.
