I'm going to wait to see what the card will eventually become under CapOne before I close mine. I never use it but I did get an CLI to 1K over the summer.
I would do everything it takes to get the AF waived and wait for a year or two. Every year try as hard as you can to get the AF waived and when it converts to Capone, call the exec office and ask the AF to be removed, the interest to be reduced, a CLI, and a PC. You might end up with a few of these if not all. Capone removed my AF and gave me a CLI to $2000 from $500. It took a lot of effort with CSR's to attempt any of this. They love telling you things like your credit limit is up to date. If my credit limit were up to date, why doesn't it match my other credit limits from conservative lenders is always my response. (They love to tell you they are a conservative lender; conservative lenders don't lend to people with damaged or short history is my response to that statement. I have developed responses to the silly things they say! Reasonable and valid responses!)
Once you get to a better card with Capone which has no annual fee then sock drawer it and use the cards that you do have which are customer friendly! Garden for a while so that your AAOA increases and move up to a better card such as Chase.