I would suggest keeping it open simply to have more options down the road. You may be able to avoid using credit forever or you may find that you wish you had built up credit at some point. I would guess the latter is more likely than the former, but who knows. But by keeping the card open and using it every now and then and making sure it's always paid on time, you will lose very little (few percent of a few dollars = a few pennies) and possibly gain a lot in future years. You also will get the deposit back without needing to close the account when the card unsecures, which may happen as soon as this fall. JMO as a former "no credit only debit lifestyle" person.
To answer the question, though, yes, the deposit is your money. If you close the card and have a $0 balance, what will happen is that they will wait a few weeks to make sure there are no delayed charges on the card, and then mail you a check for your deposit amount.
