I was in a similar situation, I had a car loan that was paid off in full, another installment loan that was paid in full, and no credit cards. I did have two collections. I opened one credit card with my credit union, once it reported to the credit bureaus (it took a few weeks to show up) my score immediately increased by 38 points. It's been about 3 months now and I've seen a steady 15-20 point increase each month after. Because you have no credit, adding a revolving credit line will be a big positive, not a negative.
