A few points:
1 - Closing an account does not drop your AAoA. All accounts on your credit report, both open and closed will impact your AAoA equally. Usually closed accounts fall off in about 10 years, so at that time your AAoA would take the drop from that account no longer being factored in. Closing your account today will have zero impact on your age of accounts factors.
2 - If a card you are considering closing has any balance, be it $600 or only $6, closing it will take that account to 100% maxed-out utilization, as the balance will be going against a credit limit of $0. This can adversely impact your score and in some cases can do so severely.
3 - Since this is your oldest CC account by a couple of years, I'd suggest keeping it open. Paying an AF though of course sucks, so I'd go with a "have your cake and eat it too" approach and contact Capital One and ask them to waive the AF forever. I'd recommend going through the Executive Office (email the CEO) rather than regular CS for this request. Simply tell them that you like the card and want to keep it, but are well aware that CO offers no-AF secured cards similar to yours now and that you'd like to be on that page. I think you could have success here. Or, perhaps they can unsecure the card for you, which could possibly eliminate the AF. I'm pretty confident that if you inquired about this that you'd come away with the AF being waived. The squeaky wheel gets the oil.
