I'm in a somewhat similar situation -- looking for another home outside Texas although in my case, cooler weather during the summer would be a consideration.
A vacation/retirement home depends on your interests in and outside golf, in my opionion. Costs may be an issue as well as surroundings, culture, lifestyle and the types of members at the location.
I've been fortunate to travel a bit and play in all the areas mentioned. I'm not much on Florida although the season there will be long. South Carolina is nice but summers can be hot. Georgia with its bugs, heat and humidity is worse. North Carolina or Virginia are both on our list but my wife would liike something that has a beach. San Diego would be ideal but most of our family is east.
I would however, respectfully counter Callaway. I'm part of a group that has gone there each year and the course has regressed in terms of conditioning and enjoyment.... in fact the RTJ Trail courses that start just across the Alabama border are much better courses. Depending on what you're life away from golf is like, I'm not sure the area has much to offer other than inexpensive living.
I've begun paying attention to the back of the WSJournal, Golf Digest and several of the club magazines one finds at country clubs to see what developments/golf communities are coming on line.
If money were no object, my choice would be Kiawah....I love several of the courses there, the food, architecture and Charleston. My wife is sorta leaning that way, but if she had her choice, she'd share a little bungelow in the Bahamas with someone who doesn't play golf.