If I woke up in your shoes tomorrow, I would concentrate on building an emergency fund and paying off the 401k loan. The gains lost (due to the loan) would bother me enough to try and get rid of it quick.
You already have a loan against one of your retirement funds (meaning you had some sort of expense and no cash on hand to withdraw) and otherwise have no type of emergency savings.
As mentioned above, I would definitely gear savings towards a general savings and not a retirement based one for more ease of access/availability and pay off the loan (frees up more to save and one less expense) and that in turn enables you to contribute more towards retirement and/or savings and less towards making up what you have had to borrow (and lost gains) and will help keep you from the cycle of having to take from it and repay again, reducing your savings potential, no matter the type.
I have been in this position before with a healthy retirement and no savings, plus a loan from 403B. Once I finally disciplined myself to budget for not only my retirement accounts, but also investment and general savings accounts, I have been able to breathe much easier. Things have come up and I have now had the ability to handle it without even considering to dip into my retirement and investment accounts and it has saved time and stress. It is still a work in progress and I have had to rebuild my emergency savings more than once, but it was there when I needed it, and I hope to continue to grow it. Now, like you, I am focusing on building a down-payment for a house in the next few years, so I am shifting savings goals and fund allocations. It can feel like a bit of a balancing act, as there is only so much to spread around. You are certainly more disciplined than I am and have definitely inspired me to trim down my budget further!
Good luck on continuing to build your future!
Is your 401(k) traditional or Roth? If your employer offers Roth, I would put at least part of your contribution into that. Then I would agree with some of the others, let that handle retirement and build an emergency fund/savings outside an IRA.
Not to say that a Roth IRA isn't good, especially if you know you will not need the funds (contribution + growth) within 5 years, but it also depends on the investments and fees, i.e. a good IRA is good, a bad one isn't! (A certified financial planner would charge you lots for such wisdom!)
I ditto the starting of an emergency fund, if it were me I'd open a HYSA and sock Money away ever pay period. If needed it's there without a removal penalty, well other than whatever transaction limits are set on the account per month. Then at the end of the year you can decide to place a portion of that amount into an IRA.