You aren't missing anything. You have to go with your gut and what's right for you. It will very rarely steer you wrong. You're doing fine.
True travel cards really only benefit people who travel frequently, as in many times per year, not just the occasional small trip. For long term use, frequent travelers get the most benefit out of it. Others should stick to cash back like you are, absolutely.
Ignore the advisor and keep doing what you're doing.
