The interest on 401K loans is normally under 10%, the money is still sitting in mutual funds earning, and you don't have to pay the government a dime.
I think you're confusing two things.
1) An unqualified withdrawal from a 401K or IRA
2) A 401K loan
However, even if one had to take an unqualified withdrawal, weigh it against how much low scores are costing one on a home mortgage loan. Unqualified withdrawals aren't always a bad idea.
Unfortunatly money that is taken out as a loan is not still earning interest. If it was still earning, taking a 401(k) loan would be the best deal in the world.
I pulled this off of Schwab's 401(k) site:
The Downside of Taking a Loan
But while borrowing from your account may sound tempting, doing so can have some definite drawbacks.
If you change jobs, most plans require that any outstanding loans be repaid immediately.
If you don't repay the loan on time, it will be considered a withdrawal from your account, which means you'll have to pay income taxes on the outstanding balance.
You may also be subject to a 10% federal early withdrawal penalty.
It's also important to understand that borrowing from your account is not a loan in the classic sense. Instead of borrowing against the assets in your account - in which case, those assets would remain in the account, working for you - you are borrowing money from the account itself. For example, if you have $20,000 in your account and borrow $8,000, your account is reduced to $12,000 until you repay the loan.
As a result, you lose the benefit of any potential tax-deferred growth on the money you borrow. If stock prices rise dramatically, for example, you'll miss out. While the interest payments you make into your account may partially offset any growth you forego, that interest is coming out of your own pocket, not from investment earnings.
Another drawback of borrowing from your account is that you will be taxed twice on the money you use to repay the loan. The money you borrow is pre-tax money. When you pay back the loan, however, you do so with after-tax dollars - money that has already been taxed once - and you'll pay again when you withdraw those dollars at retirement.
Also consider that the interest payments on your loan may put a pinch on your cash flow, which might force you to trim your plan contributions. The impact of such a move on your long-term financial security could be significant.
Finally, your plan trustee may charge a fee to set up and administer the loan.