Are you going with conventional or FHA financing? FHA shouldn't be required to pay it off, but with 20% down and your scores I'd strongly be considering conventional. Each lender has their own requirements on collection/charge-off payoff policy, if it's FHA there are plenty of lenders who won't require it to be paid, but with conventional the guidelines usually change once the balance is $2,500 - anything less it can remain unpaid and anything more it has to be paid. Other lenders have higher maximum thresholds though, it sounds like you just need to find one who has a higher limit, or perhaps ask your lender if you can just pay it down to a certain limit instead of paying it completely off.
Free Mortgage Advice & Pre-Approvals (FHA, VA, USDA, Fannie, Freddie, Non-Prime, Construction, Renovation/Rehab, Commercial) since 2002
Located in Southern California and lending in all 50 states