Measuring Housing Regulations at Scale
What AI-Generated Data Show about Rent Extraction and Exclusionary Zoning
There is widespread agreement that housing regulations have a big impact on housing supply, limiting construction in desirable areas. But tackling this problem requires us to measure and think about housing regulations at the municipal level, where they are enforced, which is an very complicated problem. The US has tens of thousands of local governments which each produce lengthy municipal codes, and so simply understanding the nature and scope of housing regulations, let alone figuring out which ones are really important and bind supply, is a big hurdle.
There are two main ways researchers tried to measure housing regulations. The “Wharton” approach, led by people like Joe Gyourko, tried to get a sense of the nationwide variation in housing regulations by sending surveys to planners around the country. This has the benefit of a nationwide scope, and has been an incredibly valuable resource for researchers, but does not always give you the granular understanding you might want.
The second approach, the “Harvard” approach led by folks like Ed Glaeser and Jenny Schuetz, tried to instead do a deep dive into housing regulations in specific areas through the Pioneer Institute. This allows for a much more precise and accurate classification of housing regulations, but is limited by the cost and expense of scaling up the approach, and was only done for 187 municipalities in the Greater Boston Area.
Together with Alex Bartik and Dan Milo, we release a heavily revised version of our paper on Generative Regulatory Measurement, along with associated code and data, which argues that LLMs are capable enough to solve this problem of accurate regulatory measurement at scale, the benefits of Harvard granularity at Wharton scale. We’ve put together a publicly available AI-generated national housing regulatory dataset and look forward to your feedback.
Generating Housing Regulations
If you upload a municipal zoning document to ChatGPT and ask some questions about housing regulations, you’ll notice that LLMs are not always accurate at the task of reading through large complicated documents and providing the right answer. Models are improving rapidly, but at this point I think it’s fair to say they do a much better job of accurately categorizing the content of a single sentence, rather than consistently picking that right sentence out of a large document. This naturally makes it hard to consistently and accurately parse regulations from lengthy documents.
We address this problem through RAG — retrieval-augmented generation, which has become standard in computer science and is just getting started in economics. The basic idea is to “embed” regulatory documents, which entails applying a mapping which transforms the document into a vector. You can think of a simple embedding as a vector the size of the number of words in the English language, which simply counts the frequency of each word in the document. Modern embeddings are more sophisticated, and vectors that are close in embedding space are more similar in meaning.
This allows you to compare the embeddings of municipal zoning documents against the embeddings of our regulatory questions, and then figure out where the answers likely lie. To get a list of regulatory questions, and put together a validation dataset, we start with the Pioneer Institute’s set of questions and answers from 20 years ago. Applying RAG, and some other tricks outlined in the paper, we think we can get pretty accurate regulatory classifications: binary questions line up between the LLM and human answers 96% of the time, while we estimate minimum lot sizes against ground truth with a 0.92 correlation on our validation sample.
These numbers are likely to only get better over time, as LLM models improve along with our ability to use them. This suggests that we can likely use LLMs to go through large chunks of textual documents — regulations, court cases, earnings call transcripts, newspapers, etc. — And categorize not just sentiment or vibes, but also their semantic content.
We do other checks in the paper, including making sure the results don’t just hold in specific geographic areas, and then scale up by collecting a large sample of municipal zoning documents to generate a national housing regulatory dataset.
What we learn about housing regulations
In the paper, we analyze this housing regulatory dataset to establish five facts about American zoning:
US housing production disproportionately happens in unincorporated areas
Housing regulation limits density. This happens through:
Bans on building or converting to multifamily buildings
Prevalence of single-family zoning
Minimum lot size requirements in single family areas
Zoning is more restrictive of density in suburban areas, particularly in the Northeast
Housing regulations can be organized into two principal components. The first associates with higher housing demand and is associated with rent extraction in Blue cities
The second regulatory PC captures exclusionary zoning
You can read the paper for the rest, but I want to focus here on the last two points about rent extraction and exclusionary zoning; which were two big things I learned from working on this.
Rent Extraction in Blue Metros
We often think of housing regulations as a force associated with housing supply. Which they are! But we also find housing regulations seem to associate with housing demand. Specifically, areas that are both expensive and have relatively high housing production have a distinctive set of housing regulations adapted to environments of extracting value.
Some of these regulations are administrative and process related: like more zoning districts and process requirements to build housing. But another typical example is mandates or incentives to build affordable housing in the form of inclusionary zoning. This is a form of redistribution of value away from developers in these areas to some residents.
Areas scoring high on this dimension share some common characteristics:
They tend to have a higher proportion of college-educated residents
They have higher job density
They have lower poverty rates
They have a substantially higher share of Democratic voters
These areas are also more likely to have retail outlets like apparel stores and dining establishments, as well as professional services including educational institutions and healthcare facilities. In contrast, they have fewer gas stations, utility services, and truck transportation businesses.
This is all consistent with many YIMBYs emphasizing standardization of process and and simplification. These do seem to be key issues in high demand cities; which otherwise actually allow density and building more than other areas.
Exclusionary Zoning
We also identify a factor which looks a lot like exclusionary zoning — suburban areas which have high minimum lot size requirements, frontage requirements, and public hearings requirements for housing. These are areas that strongly limit density and apply minimum quantity standards for housing, which have the effect of limiting entry by low-income residents, thereby exacerbating income and racial segregation. I think we all know exclusionary zoning when we see out: our goal here is to try to measure it more precisely, and connect with other outcomes.
One of the features of exclusionary zoning which really surprised me are the associations with educational outcomes: test scores are better in areas with exclusionary zoning, spending per pupil is a lot higher, as are Chetty opportunity measures — these are good places for social mobility.
I think one way to rationalize this set of regulations is the decentralized nature of school funding and administration. In many developed countries around the world, there don’t seem to be massive differences in the funding or curriculum of public schools. As a result, you don’t often hear of people in, say, Italy or France moving to specific neighborhoods to access local schools. School quality is, at least compared to the US, relatively homogenous, at least at the within-city level (there do seem to be large regional variations in some of these countries).
As a result, with less scope for educational sorting, you typically see rich people live in the center of the city. And some of these people in the city center, in places like Madrid or Buenos Aires, vote for the right wing party. This basically never happens in the US, where the rich have decamped in many cities, and city centers are invariably left-wing strongholds.
So what explains this differential sorting of people and partisanship across countries?
We can’t prove it in our data conclusively, but I think a very plausible hypothesis is that tying school quality to local conditions generates incentives for rich people to decamp for suburban areas, exclude the poor through high bulk regulatory barriers, and thereby produce enclaves of high quality educational.
On the one hand, you might think that this is the sort of “fiscal zoning” advocated by people like Fischel and Hamilton. However, it is a pretty inefficient way to get there — we have to leave large tracts of land less developed to achieve this exclusion — and it results in other undesirable features of segregation in cities.
The Midwest has some exclusive suburban enclaves — but it turns out the Northeast is really where these bulk regulations really matter. By contrast, the West regulates housing more through process. You can see some of the associations here; because municipalities typically report a range of lot size requirements, we focus on the minimum binding one:
We encourage you to use our data and see what else you can find; suggest any other housing regulatory questions you’d like us to add to our dataset, and adapt the approach to categorize other types of regulations and unstructured texts.