I paid off 4 collections (With settlements) in 3 months. All my scores increased between 49-81 points. So YES.. your score CAN go up by paying off collection/charge-off balances. YMMV, depending on how much the total balances are.
My total balances owned was $27,000.00 between 4 accounts. Got that down to 2000.00 owned on one account.
Starting Score: 521 TU, 597 EQ, 574 EX on 6/20/2011