While homeownership rates approach 50-year lows and experts say weakness in the housing sector cost the U.S. economy over $300B last year, there are parts of the country where residents will pay seemingly any price for a home. Bloomberg recently analyzed U.S. census data and determined the 100 richest towns in America, many of which are near Wall Street or Silicon Valley.

We honed in on the top four to get a better idea of the markets that house America's richest people.

1. Atherton, Calif.

Average Household Income 2015: $444,400

Median Home Price: $6.3M

Atherton, Calif., is the most expensive ZIP code in the country. That is little surprise considering the town sits right between San Francisco and San Jose and is home to many Facebook and Google execs. In 2015 a home in the town sold for $35.3M, one of the most expensive transactions in the town's history and almost making the $6.3M median home price look like a bargain.

2. Cherry Hills Village, Colo.

Average Household Income 2015: $403,500

Median Home Price: $2.6M

Denver's most affluent suburb, Cherry Hills Village is the second-richest town in America and the only town not in a coastal state to make it into the top nine. From 2014 to 2015, this area watched its average household income jump over 13% and over the past year home values have risen 8.3%. The median price per SF is more than double that in the Denver metro and is forecast to grow nearly 3% this year.

3. Scarsdale, N.Y.

Average Household Income 2015: $371,200

Median Home Price: $1.3M

Scarsdale is just north of New York City in Westchester County, and while home values here fell 4.4% last year, they are still out of reach for most Americans. Experts said there is potential for new development in Westchester and neighboring Fairfield County, but the idea of commuting to NYC turns away many possible tenants. While rents are cheaper than NYC in most of Westchester County, in Scarsdale the median rent is $5,500, or nearly double the price of the New York metro area.

4. Hillsborough, Calif.

Average Household Income 2015: $368,900

Median Home Price: $4.3M

Hillsborough shares San Mateo County with neighboring Atherton, and is only slightly less wealthy. Average household incomes dropped 5.9% from $392,100 in 2014 to $368,900 in 2015, yet home values advanced 3.1% over the last year. In Hillsborough the median price per SF is $1,127, far outpacing the $481 San Francisco metro average.