Aduling is hard! I get the instinct. I'm sitting on $90k in student loans (I only took out $50k but being in deferment/forebearnace for nearly a decade really jacked that balance up) and even on the lowest payment plan, my payment is $400 and I chose a career field to take advantage of PSLF which of course everyone is being denied. I considered not taking insurance this year to help offset that cost because I, like you, don't go to the doctor very often and am generally young and healthy. For a moment it seemed like a risk, but one that would probably land in my favor. And then a few days before I was set to make that move, my doctor found a health issue that ended up taking dozens of appointments, countless expensive scans, and tens of thousands of dollars to resolve. Thankfully, my insurance covered most of it.
Moral of the story? It isn't worth the risk, even if your employer doesn't demand repayment. Health issues sneak up on you and you do not want to be caught without coverage.
