With rents skyrocketing in San Francisco, affordable housing activists’ ire has turned to the region’s tech industry. Tech buses shuttle workers from in-city neighborhoods to suburban office campuses in Silicon Valley, and have been the most visible target, with regular protests in front of the stops (some of which have even turned vomitous.)

Tech companies — most notably, Twitter — haven’t helped by accepting millions in tax breaks, but the blame for San Francisco’s stratospheric housing costs lies not with giants like Google and Facebook or the smaller startups, but with Silicon Valley itself — the string of wealthy towns lined up along the Caltrain commuter rail tracks down the San Francisco Peninsula, in San Mateo and Santa Clara counties, which have steadfastly refused to match torrid economic growth with anything resembling a commensurate amount of housing and office space.

The whole of San Mateo County, which includes highly desirable, wealthy towns like Menlo Park and Atherton, only added a measly 10,455 houses and apartments over 10 years to its existing 2000 stock of 260,576 — barely a 4 percent growth rate. Though San Francisco certainly shouldn’t be let off the hook for not keeping up with housing demand, at least there the housing stock jumped by 8.8 percent from 2000 to 2010, according to the decennial censuses. The growth rate didn’t quite keep pace with national population (9.7 percent) or housing stock growth (13.6 percent), but certainly it beat the surrounding ‘burbs.

Santa Clara County did better, building an additional 52,591 units on top of the 579,329 it had in 2000, working out to 9 percent. But the growth was disproportionately located in the sprawling southern and eastern portions of the county, rather than in the denser, more transit-accessible and in-demand northwestern parts. Towns like Palo Alto and Mountain View, now household names thanks to their tech offspring, where demand for office space and housing is most acute, grew much slower. Tiny Palo Alto grew its number of homes by 8.3 percent — not enough, but not terrible. Mountain View, on the other hand, home of Google, added just 715 new houses and apartments, yielding a pitifully low 2.3 percent growth rate in the first decade of the century.

The disparity in growth hints at an ugly tension in housing growth in Silicon Valley, and throughout the Bay Area in general: The places that are allowing construction are relatively poor places that are most likely to see gentrification because of it, while the already wealthy places with the highest demand have walled themselves off to growth.

Redwood City, for example, is the among the poorest Peninsula cities along Caltrain, with a median income of just $30,000 in the surrounding census tract. It also perhaps the most development friendly. A new downtown development plan is slated to allow up to 2,500 units to go up in the city’ center, and half have already been approved.

“We saw no opportunities to do multifamily in say, Menlo Park,” said one developer, “because the city is not working to allow the densities we need.” Redwood City, on the other hand, “has developed a precise plan for their downtown core that allows densities that justify paying upwards of $200 per square foot for land because you’re able to get significant densities on fairly small parcels.”

Compare this to Atherton, just one stop down the Caltrain, where the median household income is the highest in the U.S. and no new housing is planned. Or look at the developer’s example of Menlo Park, home to Facebook. There, a group called Save Menlo is working toward a ballot initiative that would cut the amount of allowed new office space nearly in half, and they’ve won the support of a number of different former mayors.

Already at least one property owner has thrown in the towel, selling his development site — which he’s been trying, unsuccessfully, to build on for nearly a decade — telling the San Jose Mercury News that as he seeks a replacement site, “one of the requirements I put on myself is, ‘I will not buy in Menlo Park.’ “

In San Francisco, a similar dynamic is at play. Neighborhoods like the South of Market area and the former port lands in Bayview-Hunters Point are seeing a huge influx of development — which is largely justified, given the obsolescence of inner-city industry and wide-open tracts — while much wealthier, more in-demand residential neighborhoods north of Market Street (like the Marina District, where there are plenty of structures without historical value that could be redeveloped) see much less new construction.

It’s areas like Menlo Park, Atherton and the Marina that are most in demand, and where zoned capacity would be most quickly eaten up by developers. And it’s these areas where development will not spur gentrification, since they’re already about as desirable as they can be. On the other hand, development is always harder where there are neighbors to complain, and especially rich neighbors, as would-be builders in Menlo Park are finding out. But if San Francisco and the rest of the region are ever to see housing prices come back to earth — or at least stall their dizzying climb — without the widespread displacement that new construction and its associated amenities bring to poorer neighborhoods, these are precisely the areas where development must occur.