Average age counts the age of both your open and closed accounts. The closed accounts stop counting 10 years after they are closed and at such a point, if you don't have accounts near as old you could take a point hit from losing an account off your report.
I believe FICO also factors the ages of your various types of accounts (installment vs revolving) though I am not quite sure how that calculation cuts. Many of the pros on here can perhaps offer you more detail.