Never seen any other Belgians on this board before, and now i spot two

As far as i'm concerned guys like Poulter are clowns. There's a fine line between smart dressing and borderline costume parties. The look you described (kahki pants, polo shirt) is pretty much the same over here.
The main difference might be brands like Galvin Green which (or so i understand) are difficult to come by in the states.
It is indeed still seen as an elite sport. Clubs make a lot of effort to get rid of this stigma, for example initiations and welcome packages have been very popular the past few years. But there is also a shortage of clubs. I played the biggest club in Antwerp a few weeks ago, Cleydael, and members were complaining about the amount of members. Weekend competitions spread over 2 days, so no tee times for members who want to play casually. Tee times for members have to be booked more than 1 week in advance, if they sell those times as business packages you're screwed basically and loose your tee time.
I do hope more courses will be build in the future. Club memberships have been rising over te last few years, so the shortage might drive the prices up
BTW: the Lindeberg shirts are very comfy and very well made. I have a plain white one and a plain navy blue one, so nothing special.