Why Does Everyone Think I Love Golf So Much?

Sure I wear golf shirts to work nearly every day; sure I could break down the Ryder Cup possiblities; so maybe I do get two golf magazines; maybe I do hold fairly strong opinions in terms of Tiger vs. Jack, but why--I ask quite discouraged--does everyone think I like golf so much?
All these things have easy explanations: I hate to shop and people buy me golf shirts--so I wear them; the golf magazines were also gifts; I like to watch Tiger because watching him is watching history unfold.
I play under ten times a year--only when invited by somone. Never do I say to myself, "Boy, I would really like to play golf today." Yet whenever I go visit family or friends, someone says "bring your sticks." Ugh. Just what I want when travelling with my family, on top of the car seats and multiple bags--that my kids can't carry--I think I should bring my golf clubs. What fun.
For me, vacation and time off are time to spend with my wife and kids. They still all like to have me around, so why run off and spend five or six hours away from them? Got me. If the kids end up liking golf--they love mini golf and the range--I suppose they can drag me out on the weekends. I might even start liking it more and actually try to get better.
My parents and grandparent taught me the game; I grew up caddying and played a lot when I was little. I can even put together a couple of good shots and good holes to this day, but if I never played again, I wouldn't care. I just don't get what's so dang great about it and am quite tired of everyone wanting me to join their obsession.