No credit card required
Browse credit cards from a variety of issuers to see if there's a better card for you.
Yes, I'd recommend calling them, figure out whether or not it's yours, and if it is yours I'd ask if they can get you started on a payment plan. The goal would be to either (a) ask them to "rehabilitate" the loan, meaning that after x number of on-time payments, they drop all the negatives and report it as an account in good standing (this is likely a harder outcome to achieve, but would be ideal since you'd get to keep a positive tradeline on your account) or (b) ask them to agree to delete their reporting of the account entirely, either once it's paid off or after you've made a certain number of on-time payments. The key is whatever you agree to on the phone, get it in writing (even if just an email to confirm the terms). This should protect you if for any reason they don't do what they agreed to do (which may of course just be an oversight on their part, but still gives you good documentation for any follow-up conversations).
And yes, there is the risk that you'll find out it's yours, and learn that all they want you to do is pay it off - with no concessions on their part. I just think as an alum of the school it's really unlikely they'd be that harsh, but nothing about this is certain and that is a risk you'd have to be willing to take if you pursue this approach. I still think that risk is better than the alternative - letting it sit on your account - especially if it does prove to be a federal loan, which could ultimately lead to either interception of your tax refund or a lien on your assets.
If you start paying, and they don't do anything extra to help you, then as RobertEG mentioned your individual late payments will fall off as each one becomes 7 years old. So, I'd look at the most recent negative entry they have on this account, and count from there (say the most recent late payment logged is 10/2011 - it would then come off your report on 11/2018). And as time goes on, and the only lates left are 5 or 6 years old, they hurt your score a lot less - so you're still making progress, even if slower than you'd like.
You need to call Columbia and figure out what the "student loan" actually is. I have a feeling that it may not be a Perkins/Stafford or other federal loan, but it may be some institutional credit. I attended a school in NJ and left with a $1500 balance for housing chargers, when the collection agency called they said it was a student loan which it was most certainly not. It was actually a "promissory note" from my monthly payment plan that was separate from expenses covered by loans/scholarships/grants. Thankfully it never reported to the CRAs but my state refund was offset two years in a row. The first collection agency was impossible to deal with so I called the school and they made the collection agency accept the payment arrangement that I offered. They were super helpful and in no way judgmental. FWIW, I consolidated a few years ago and for some reason one loan remained outside the consolidated loan so I have to pay that separately.
You can also look up all your student loans in the National Student Loan Data System to see which company is servicing them. http://www.nslds.ed.gov/nslds_SA/
Wanted to update the status of this, in case it applies to others ...
My unpayed loan was, in fact, a Perkins loan.
Once I started researching Perkins loans, I discovered that schools MUST offer a rehabilitation program for Perkins loans (and several other types of federally subsidized loans) that will allow you to make nine consecutive payments and then have ALL negative information about the loan removed from your credit report. Once you've made the nine payments, your old promissory note kicks in and you are back to paying off the loan as if you never defaulted.
Some schools have an explanation of the rehab program on their websites. So take a look. Details would also be included in your promissory note. This document, a guideline for lenders, was very helpful, too.
Worth noting: the finance department had been sending mail to an address where I have not lived in four years. They claimed they never checked with the school I attended for an updated address -- which they have to do, by law -- because they didn't get any returned mail or notification that the address was bad. If the rehab program weren't so simple, I probably would have involved a lawyer to push on the fact that they did a lousy job of finding me. I can't believe the school would pay collection agencies who can't bother to find a fairly public person.