Finding and Fixing the REAL Housing Crisis in California.

Executive Summary 

1. We're solving the wrong crisis. 

The real issue: Our ratio of jobs-to-housing units suggests metro areas in California have a healthy balance of jobs and housing units.  However, there’s a serious mismatch between the type of housing and the type of job. California has far too little affordable housing for the minimum wage-earners who live in metro areas. Minimum-wage earners, in large numbers, are forced to either stretch well beyond their means or combine households to pay market-rate rent in places where they live and work. That’s the crisis.
 

For the record, the Embarcadero Institute takes the real housing crisis seriously and is committed to measures – whether taken at the federal, state, or local level (and ideally there’d be a coordinated effort by all three) – that would turn the tide of longstanding problems of housing inequity. 
 

2. Doubt and uncertainty have politicized the problem and the solution.

State legislators, such as Senator Wiener, have undermined the work of the Dept. of Finance (DOF) and created doubt and uncertainty about its estimates of housing needs. This has paved the way for other special interests to question the targets. Big Tech and real estate interest groups benefit when incentives are created for market-rate housing, but none of that helps affordable housing. According to California State Lobbying Search, tech-backed CA YIMBY spent more than $1.2 million lobbying for bills that favored market-rate housing. The state should realize the market will take care of itself, and focus on the real challenge – providing sufficient housing for its most vulnerable workers and residents.

3. Only two estimates clear the state's feasibility hurdle.

It turns out there is a limit to how much housing is good for the economy. Given the number of jobs that exist in a labor-market area, there is a ‘goldilocks’ ratio for the number of housing units relative to jobs. If too few housing units people have to commute in for jobs from housing that is outside the metro area, but if too much housing the problem is reversed, i.e., too many people have to commute out to jobs. Neither situation is great for fighting climate change. Two housing estimates, both from professional demographers, produced a healthy balance of jobs and housing, according to the American Planning Association: 1) the Dept. of Finance estimate, that does not include the second round of adjustments called for by Senate Bill-828, and 2) the Federal Home Loan Mortgage Corporation (Freddie Mac) estimates.

4. Clarifying the cause clarifies the solution.

NIMBYism isn’t the cause of the affordability crisis. That’s a convenient but unsubstantiated deflection. The affordability crisis is a result of the state’s deliberate abrogation of its responsibility to provide housing for lower-income families. For decades the state has set a target of one affordable home for every one market-rate home but has failed to provide funding for those affordable housing targets. As a result, the Bay Area has for decades been building one affordable home for every four market-rate homes, and more recently only one affordable home for every seven market-rate homes. After several decades no trickle-down housing has materialized. Instead, lower-income households have been either forced to stretch well beyond their means or combine households in order to pay market-rate rents. 

The solution: the state either needs to commit to a state housing voucher program to supplement federal efforts for lower-income families, or it needs to prioritize a statewide, state-funded effort to build affordable housing. If solving the housing crisis is the state’s number one priority as many state legislators and the governor suggest, then the budget should reflect that. Instead, we have seen state funding for affordable housing gouged in 2010 with the shuttering of Redevelopment Agencies and never replaced. In addition, Governor Newsom has twice vetoed thoughtful bills that would have increased funding for affordable housing. It will only become a more expensive problem to solve going forward, as increased density drives up the price of land.
 

5. To our critics: Please read and check before opining.

Housing needs analysis is complicated and careless readers of our reports keep suggesting we said things we didn’t. We’re a little tired of guys with degrees from elite colleges who either aren’t able or can’t be bothered to follow the arguments we make with government data and then misrepresent our work. You can read more below, but in summary here’s what the Embarcadero Institute believes.  

1. We believe the analysis of California’s Dept of Finance. Unlike some state legislators, we think those guys really know what they are doing.

2. Adding fudge factors to the housing numbers doesn’t help solve the problem. It actually hurts because a) it camouflages the real problem – that we have a very serious shortage of affordable housing; b) it leads to sloppy, unsubstantiated magical thinking such as the state's 60% affordable housing target will materialize if we just build more market-rate housing;  c) it provides cover for Big Tech and real estate interest groups to push their profit-seeking agendas at the expense of lower-income families.

3. We think the state needs to accept responsibility for its role in under-funding affordable housing for decades. The state should stop blaming cities for the fiasco they created. Cities are exceeding their state-mandated historical market-rate housing targets. They’re only failing their affordable housing targets because they don’t have the funding to make affordable housing projects economically viable for developers. The state not only doesn’t support them in their efforts to build affordable housing, it undermines them by creating more incentives for market-rate housing.

One of our detractors, Professor Elmendorf (UC Davis), has published his own proprietary model to estimate housing needs in the Bay Area. His model is the basis for a tech-backed YIMBY Action lawsuit against the state Dept. of Housing and Community Development (HCD). In reviewing his work, however, we found a number of errors. He used simple averages where HCD had used weighted averages. In other cases, he used only five instead of all nine Bay Area counties, and may have selectively excluded certain qualifying "fast-growth" metros from his data set. It also appears he may have used only owner-occupied instead of owner-occupied and renter data in his cost-burdening statistics. The red flag for Elmendorf should have been that he couldn’t reproduce the HCD numbers (the Embarcadero Institute could). Instead of figuring out where his analysis went wrong, he assumed he was right and that the error lay with HCD. It begs the question (and harkens back to our point about state legislators undermining the work of their own departments) how it can be that a proprietary housing assessment with data errors ends up being the basis for a lawsuit against the HCD.

Now, with more detail

1. We’re solving the wrong crisis.

Fear, Uncertainty, and Doubt (FUD) have unfairly undermined the credibility of California’s Dept. of Finance. And just as Twitter poisoned the national political conversation, so too Twitter has fueled an echo chamber of YIMBY rage based on the housing FUD. Much of this has been driven by mostly white well-educated younger men who complain they’re paying too much rent, even as affordability data suggests they’re doing just fine. The empathy for young professionals is fueled by journalists like Ezra Klein and others on Twitter who consistently refer only to rent when discussing the housing crisis. They forget that affordability has a denominator – income. Sure, median rents are crazy high in San Francisco, but so are median incomes. The data show young professionals aren’t in crisis (arguably there’s no better place for them to live on their residual incomes). 

Fig 1.  Median Residual Incomes by Metropolitan Statistical Area.

Unfortunately, the entitled rage on Twitter is sucking the oxygen out of the real debate that needs to happen. How do we make housing more affordable for everyone below median income? Because that is where the crisis fairly and squarely exists.  For decades the state has demanded cities build one affordable home for every market-rate home, but the demand came without financial assistance. Instead, because of a lack of state funding and support, the Bay Area has been building one affordable home for every four market-rate homes, and in the most recent housing cycle, one affordable for every seven market-rate homes.

Fig 2.  The ratio of market-rate to low-income housing permits 1999 to 2019.

This hasn’t resulted in any trickle-down. Instead, it has resulted in lower-income households now forced to step up and double-up to pay market-rate rent. After decades of state neglect, we have far too little affordable housing for the number of minimum wage earners in our metropolitan areas. 

Fig 3.  The Match between Low-income Jobs and Low-Income Housing (2017)

2. Doubt and uncertainty have politicized the problem and solution.

One of the challenges to understanding the causes of and solutions to the affordable housing crisis in California has been the continual dismissal of the work of the Dept. of Finance (DOF) by state legislators. They dismissed the work of the DOF when they embraced McKinsey’s estimate of California’s housing need (3.5 million by 2025) over the DOF’s (1.2 million by 2022 - from the state-wide 5th housing cycle targets). They dismissed the DOF again when state legislators passed Senator Wiener’s Senate Bill 828, a bill that assumed that the DOF had gotten their household projections wrong. Senator Wiener, without evidence, claimed the DOF had underestimated the housing need and had failed to account for existing housing needs (not true). Senator Wiener in his bill, SB-828, authorized another state department, HCD, to make a second round of adjustments to the DOF estimates. Using fear about the housing crisis, Senator Wiener managed to create uncertainty and doubt about the DOF’s work. This standard propaganda tactic (FUD) paved the way for others, like Professor Chris Elmendorf, a Law Professor at UC Davis, to challenge the work of the DOF and HCD with his proprietary housing model.

Two methodological approaches emerged from the housing estimates debate.

On one side there are the professional housing economists and demographers at California’s Dept. of Finance whose work comports with work done by professionals at the federal banking regulator, Freddie Mac. Both models produce housing need estimates that are consistent with standards set by the American Planning Association. Housing needs estimates from the professionals are on the lower side of the divide.

On the FUD side, there’s Senator Wiener (the largest recipient of real estate special interest money in the state senate); Professor Elmendorf, a law professor; CA YIMBY (heavily backed by Big Tech); and McKinsey. The proprietary models of Wiener (SB-828), Elmendorf, and McKinsey don’t follow any peer-reviewed methodology and their results range from double to triple those of the professional models.
 

3. Only two estimates clear the state's feasibility hurdle.

Elmendorf recently raised a good question – who decides whether California misjudged the Bay Area’s housing needs? It turns out the answer to that question is in Govt. Code Code 65584.01 (c)(1). The law requires:

“that the existing and projected housing need, as established, must achieve a feasible balance between jobs and housing using regional employment projections from the regional transportation plan”

The American Planning Association (APA) states the optimal balance between jobs and housing is in the range of 1.3 to 1.7 jobs for every housing unit. The Association of Bay Area Governments (ABAG) claims the optimal ratio for the Bay Area is 1.4.  By the APA standard, only the housing needs estimates from the HCD methodology before SB-828 became law (shown in chart as "HCD Pre SB-828") and “Freddie Mac” achieve a feasible balance between jobs and housing for the Bay Area. The proprietary models all lead to unhealthy jobs/housing ratios. Elmendorf’s estimate leads to a number that is historically associated with a market crash.

Fig 4.  Existing Jobs-to Housing Ratio (including existing housing needs)

Note: All Housing needs estimates include the DOF adjustment for existing overcrowding and cost-burdening which is part of the base they all build upon.

* Uses 2019 jobs to avoid reflecting job loss due to COVID since the housing needs assessments were made pre-COVID conditions. Source: CES rather than QCEW employment numbers since CES estimates also cover self-employed, domestic workers, proprietors etc otherwise not covered in QCEW data.

4. Clarifying the cause clarifies the solution.

The debate about whose numbers are right has also fueled a debate about the root causes of the crisis. Journalist Conor Dougherty would have you believe that NIMBYism is the “leading cause” of rising rents in California and beyond. He thinks that if residents would only stop resisting housing density the problem would be solved. In 2019, California cities approved more than 116,000 housing units. How many housing units does Dougherty think did not get approved because of local housing resistance? If it’s the leading cause, and he thinks California is short 3 million homes, we have to assume he’s thinking hundreds of thousands of shovel-ready homes with financing were victims to NIMBYism. Really? Does that seem plausible? Where’s the quantifiable evidence for this meme? 

On the other hand, we have Federal Reserve economist, Jonathon Rappaport, who writes in his 2018 paper,  The Faster Growth of Larger Less Crowded Locations:

“In particular, population density accounts for more than twice the variation in median home prices across medium and large metropolitan areas”

Rappaport's findings suggest What if people like Dougherty are wrong and housing density isn’t the solution to reducing rents? What if housing density happens to be the most important factor driving up rents? To be crystal clear, the Embarcadero Institute is not arguing against density. We’re simply saying density comes at a price – a price that can be paid by the educated middle class with highly sought-after skills. Their salaries will rise to meet the rising rents. The much-criticized NIMBYs will also be fine – their homes will just continue to become more valuable. Lower-income workers, not so much. And that’s the problem that needs to be addressed by the state government with our tax dollars.

So while Dougherty stirs up intergenerational warfare because let’s face it, journalists love a good controversy, especially one with a human element, he lets the real culprit off the hook. The state government of California. 

The state didn’t build the necessary affordable housing when the land was cheaper, now the state is either going to have to create a state housing voucher program for lower-income families or step up and build the AFFORDABLE housing they should have built decades ago. Only now, it’s going to be a lot more expensive. This is not the fault of cities and their residents. It’s the fault of a state that has abrogated its responsibility for its less fortunate residents while prioritizing the special interests of Big Tech and real estate. California can still pull itself out of this mess. It has the funds.  It has the highest tax revenue of any state – $188 billion dollars (2019) (more than double the 2nd place state), and it has the highest tax revenue per capita of the top ten largest states. If any state has the resources to fix this, it’s California.

We’re discouraged that the Big Tech-backed YIMBYs shy away from these causal discussions. We’re dismayed that, rather than engaging in a discussion of the facts, like the SB-828 double count, the YIMBY apparatus has doubled down on fear, uncertainty, and doubt, and lawsuits fueled by a law professor’s housing model with math errors. None of this bodes well for finding real solutions to help the people most in need. 

Note to Texas: if you want to avoid the folly that is California, ruthlessly prioritize the building of affordable housing. Like yesterday.

5. Now, to address our critics. Please read and check before opining .....and publishing.

The Elmendorf model and errors unpacked.

Chosen Benchmarks

Elmendorf disagreed with HCD’s choice of regional benchmarks. He argued that it was wrong to benchmark against slow-growing Combined Statistical Areas (CSAs) like New York, Boston, Chicago, and Denver and that the HCD was essentially ‘baking in' slow growth. He argued that the HCD should be benchmarking California’s urban areas against fast-growing metros – Metropolitan Statistical Areas (MSA) that had grown more than 30% between 2000 and 2013 – sprawling metros like Atlanta and Dallas.

However, Elmendorf’s regional benchmarks had a few issues of their own. For one thing, Metropolitan Statistical Areas (MSAs) are a smaller geographical unit than the Combined Statistical Areas(CSA) used by the HCD. Elmendorf overcame this by using the CSAs that corresponded to his chosen list of fast-growing MSAs. However, two of his MSAs didn’t have corresponding CSAs, and none of the corresponding CSAs met his original target growth of 30% between 2000 and 2013. 

Separately, his choices of benchmarks, while fast-growing, are also far less dense than the California urban areas. Senior economist, Jordan Rappaport, from the Federal Reserve Bank of Kansas City put it best in his December 2018 paper, The Faster Growth of Larger, Less Crowded Locations:

“growth across medium and large metropolitan areas is strongly negatively correlated with population density.” 

With that in mind are Elmendorf’s benchmarks really more appropriate?

Fig 5.  MSA growth (as measured by housing permits issued between 2000 and 2013) versus Density for top 70 Metropolitan Statistical Areas

Cost-burdening 

Elmendorf argued that the cost-burdening adjustment shouldn’t just be applied to future housing but should be also applied to existing housing. That would be a reasonable argument if you didn’t know that the Dept of Finance had already adjusted for this. 

Elmendorf was unable to reproduce the work of HCD. He couldn’t reproduce its cost-burdening or its overcrowding statistics. He assumed HCD had gotten their numbers wrong and that he had gotten it right, but he was mistaken. The Embarcadero Institute reproduced all of HCD statistics, and as a result, we were able to identify some, but not all, of the errors, Elmendorf made in his work.
 

Firstly, Elmendorf used simple averages when the HCD used weighted averages. 

• The reason the HCD used weighted averages is that overcrowding is reported as a percentage and the regions being evaluated are significantly different in size – the largest being six times the smallest. In such cases, a weighted average approach is recommended. Elmendorf, however, assumed that “the comparators were equally weighted” in the HCD model. They were not, nor should they be.

Secondly, Elmendorf may have used only owner-occupied data, accidentally excluding renter data, from his cost-burdening analysis. It’s the only explanation we can come up with to explain why his numbers are so low. 

• From the same Comprehensive Housing Affordability Strategy (CHAS) data set, HCD found that 66.6 % of lower-income households (extremely low, very-low, and low) are cost-burdened in the Bay Area. The Embarcadero Institute concurred with the HCD findings. Elmendorf reported the number to be only 58.5%. Fortunately for Elmendorf, whatever error he made, he applied it consistently across all his data, minimizing the impact of his error. As he reported himself:

 “Our calculations came out a little differently but the bottom line is very similar.”

It’s fortunate the final outcome is somewhat “similar,” but it would be helpful to understand how Elmendorf came up with his numbers, particularly since his report is to be the basis for a lawsuit against HCD.

Jobs-Housing

“data regarding their assumptions about the relationship between jobs and housing, including any imbalance between jobs and housing”. 

Elmendorf interpreted this to mean that, not only did they have to supply these data, but that HCD also had to use these data to create an additional adjustment in their assessments. Elmendorf decided the best metric to measure the job/housing imbalance was the number of 90-minute long commutes in the region. Elmendorf took the number of super-commuters (commutes 90 minutes or more) and divided it by ABAG's accepted jobs to housing ratio (1.41) to determine the additional housing units needed. Unfortunately, for Elmendorf, he only used data for five out of the nine Bay Area counties in his calculations. When this is corrected his jobs/housing adjustment is increased. 

It’s interesting that Elmendorf didn’t choose the standard measure for the jobs and housing relationship – the “jobs-to-housing” ratio. By that measure, the region, with its approximately 4 million jobs and 2.9 million housing units, has a ratio of jobs-to-housing of approximately 1.38, inside the American Planning Association’s healthy range which is 1.3 to 1.7, and actually slightly less than ABAG's healthy standard.

Common Sense Prevails

Elmendorf was eager to point out the HCD’s overlooking of government code but he appears to have overlooked some of the code himself. A little further down in Govt. Code 65584.01 (c)(1) there’s a requirement

“that the existing and projected housing need, as established, must achieve a feasible balance between jobs and housing using regional employment projections from the regional transportation plan”

When we tease out the part of Elmendorf’s estimate that relates to existing rather than future housing needs, the number is approximately 573,000 housing units. If that amount of housing were added to today's housing supply, his jobs-to-housing ratio would be 1.15

                        
                                               *see notes from Figure 4 above.

Take-away:

Elmendorf should correct the errors in his paper. He might consider some peer review before he publishes next time.

Combined Statistical Areas used as benchmarks in HCD and Elmendorf models.

Table for housing estimates in Figure 1.