To be honest, you are probably seeing an economic divide. Traveling is expensive, and getting a passport is expensive ($135, put in the perspective that it's a week of groceries for a small family). Those who have the means travel, those that don't, don't.