I paid off 4 collections (With settlements) in 3 months. All my scores increased between 49-81 points. So YES.. your score CAN go up by paying off collection/charge-off balances. YMMV, depending on how much the total balances are.
My total balances owned was $27,000.00 between 4 accounts. Got that down to 2000.00 owned on one account.
Starting Score: 521 TU, 597 EQ, 574 EX on 6/20/2011 Current Score: 753 TU, 764 EQ, 766 EX on 02/17/2014 Goal Score: 720-740 Across the board