Author: bgoggin

Continuing the recent line of posts about SF residential development, I’ve finally decided to aggregate residential completions over the past 3 1/2 years rather than look at present development. While not surprising, the results are stark: SF housing additions are extremely concentrated in a handful of areas, particularly the South of Market area.

I have also used this post to experiment with the geopandas package in python. I am a complete novice and ran out of patience for some of the legends for these graphs, so apologies for their quirks. Hoping to update this post when I’m more experienced!

Onto the data…first, let’s plot dots of completed construction projects in the past 3 1/2 years. In particular, these are residential completions in the SF development pipeline from Q4 2012 – Q1 2016 because this is the only period for which there is consecutive data. The graph shows what we’ve seen in my previous posts of recent development: spread out, but with much larger developments on the eastern side of the city and, in particular, along the Market Street corridor. In anticipation for my next two maps, I overlay this data with neighborhood boundaries for San Francisco from Zillow. That one blue dot in Bayview has negative 1 units added. I sadly have not yet found a good way to integrate it into the legend.

Next, the moment we’ve (read: I’ve) been waiting for… let’s aggregate by neighborhood! For those interested, I do this by performing a spatial join in geopandas between the two layers (both projected into UTM zone 10).

While not totally unexpected, the map is startling. Pretty much all of the residential additions in the past 3 1/2 years are on the eastern half of the city, and SoMa has added more than 3 times the amount of units as the nearest area. Moreover, areas like Richmond and the Sunset District have added less than 50 units each! Although the housing supply is growing with demand, not every area of the city is sharing this burden. It’s important to point out that this is due to deliberate zoning decisions made by the planning department in the past 10 years. However, behind every planning decision is tremendous political pressure from neighborhood associations and the like. This development pattern can ultimately be traced back to NIMBYs.

Finally, let’s look at affordable units by neighborhood.

In general, the same areas building any units are the same ones building affordable units. But look at all the zeros! Clearly affordable housing is not an enticing prospect for most of the city. SoMa, I declare you the winner.

Following up on my post a few weeks ago about the state of current residential development in SF, I decided to focus more narrowly here. In particular, above I have mapped the residential developments that were completed in 2015, broken down by whether a development has affordable units or not. In technical terms, affordable units are defined as those for which rent equals 30% of income of a household with an income at or below 80% of San Francisco’s “fair market rent” (defined by HUD). In other words, they have some sort of rent ceiling. While I do not distinguish by number of units on the map, this information is available for each development by clicking on it.

Surprisingly, there are not many clear spatial patterns here. Development—both market rate and those with affordable units—appear to be fairly uniformly distributed throughout the city. In future posts, I hope to map this development for multiple years to explore the broader context of development.

Data comes from the 2015 Housing Inventory Report by the SF Planning Department.

For all of the talk surrounding the Bay Area’s housing crisis, I have found it difficult to understand not only the scale, but also the geography of the problem. Although we can get a decent sense of where development is happening via Census data on housing, this is at best polygons at the county or MSA level. The housing crisis is a problem at the most local levels of government. While the market would probably supply more housing if allowed, local land use controls at the municipal level have time and again blocked development.

So where can we find data on residential development at the municipal level? This is much harder than I initially thought. Most cities do seem to collect this information in their planning departments, but few have this information publicly available. One of the few planning departments that does publish such data is San Francisco’s. Through what’s known as their development pipeline reports, we can identify and map all development projects in the city that would add residential units or commercial space. This consists of all developments which have at least submitted applications to the planning department or department of building inspection. Furthermore, this data excludes projects that do not add residential units or commercial square footage (e.g. roofing upgrades). Within this data, we can group projects by different stages in the development process, such as planning review, building permit review, or construction. Furthermore, we have information on the number of net units added, which can be negative if a project adds less development than the previous residence had.

To start with, let’s take a look at a map of current projects by net units added. This data comes from the development pipeline’s second quarter 2016, which is the latest data available. For all intents and purposes, we can consider this the current state of new housing development in San Francisco.

A few notable points. First, the South of Market (SoMa) area stands out for both the overall quantity of development and the size (in terms of net units) of those developments. This has not gone unnoticed, as some have even called SoMa the “next phase of San Francisco’s urban history.” Second,  the overwhelming majority of new residential development (over 80%) is less than 50 units. Although large luxury apartment buildings in SoMa may grab the headlines, these are definitely outliers in a city that is still mostly zoned for low density. Thirdly, there is a lot of development in low-density areas of the city (e.g. Richmond or the Sunset District) that have relatively restrictive zoning. People don’t seem to shy away from building there even if small developments are all that is allowed. Finally and perhaps most importantly, look how well the map above corresponds to a map of the zoning code! Pretty much all of the big developments seem to be outside of standard residential zoning.

Overall, the above points signal that the market is not the problem. At first glance, residential development seems to be happening fairly uniformly across the city. On the other hand, there is tremendous regulations as to how this development can take place. At least in terms of units per development, zoning is king.

 

As many in the city planning twittersphere may know, the Bay Area Rapid Transit (BART) system’s twitter account has, from time to time, engaged customers in lively debate. In one particular instance earlier this year, BART responded to customer complaints over track maintenance that the system has had an unpredictable rise in ridership coupled with a lack of investment.

[ref]Screen shot taken from Upout.com (http://www.upout.com/blog/san-francisco-3/barts-twitter-account-getting-insanely-real-issues-lately)[/ref]

Leaving aside the state of BART’s infrastructure and investment, I want to take a closer look at BART’s ridership here. Exactly how much, and in which areas, has BART ridership increased? This simple exercise can help transportation planners determine both which stations serve the most people and where the ridership trends are heading. This, in turn, can serve as a metric to allocate limited resources. While this limited analysis does not fully answer these questions, it exposes some interesting trends to guide the inquiry, along with providing compelling evidence in favor of BART’s twitter defense.

First, let’s look at average weekday exits —defined as a single instance of a rider exiting a station—in 1999. I concentrate on weekday exits here because, anecdotally, workdays seem to generate the most customer frustration. While being late to a Sunday brunch is embarrassing, tardiness to a board meeting is probably worse. Furthermore, weekdays are when ridership is highest. As for 1999, this is the earliest year that I can find from BART’s weekday exit records. However, this works well because it roughly approximates the beginning of the tech boom mentioned in the twitter fight above.

Unsurprisingly, downtown San Francisco stands out as the single outlier on the map. While most stations in 1999 had around 5,000 exits per week, the Embarcadero and Montgomery stations downtown both had over 25,000. Also, the East Bay has fewer exits than the southern outskirts and of San Francisco—emphasizing that this area was the dominant commuting corridor at that time. The only exceptions to this are downtown Berkeley and the two downtown Oakland stations, all three of which had around 10,000 exits per week. This evidence certainly suggests that there was a relatively manageable set of heavily travelled stations. What does the map of 2016 exits look like, you ask?

Ridership grew almost everywhere throughout the system. Traditional heavyweights such as downtown San Francisco, Oakland, and Berkeley grew even bigger, with the top two stations—Embarcadero and Montgomery stations—now accounting for over 45,000 exits per weekday. Perhaps most importantly to the service quality of BART, the outskirts of major cities and suburbs grew sizably (with several new stations appearing). Not only could this put more strain on an aging system, but it also spreads track use over a wider area, making quality of service more and more difficult to maintain.

Finally, what do the numbers say in percentage terms?

 

Viewed through this lens, downtown San Francisco is far from king of the system. While there is still a wide gap in terms of absolute number of exits, Oakland, Berkeley, and the East Bay suburbs all equal or exceed SF in growth of ridership. Also, the outlying areas of the system are booming! More and more riders are taking longer rides, which surely puts strain on the system. Whoever is in charge of the BART twitter account was not lying. Riders may want to take it easier on a system that, built 60 years ago, has experienced dramatic growth in ridership over such a short period.

 

Data Sources and Decisions

Average weekday exit data by station comes from BART’s Ridership Reports page, found here. Additionally, geospatial data (kml file format) for the BART system comes from the BART website here. Unfortunately, matching on a string name variable was necessary to merge the geospatial data with the weekday exit data. There is a python script that includes a dictionary for this matching process at my github page here. In sum, I export a csv file with a crosswalk between the two datasets and then merge them within Carto. Also in the python script, I generate the percent change variable used in the above figures.

I made a number of data decisions within Carto during the mapping process. First, I chose a dark basemap and bright colors for the stations for contrast. I also made sure that the basemap’s labels do not overshadow the station points layer. In order to visualize the data, I chose a bubble map to compare an attribute—namely, station exits—of a points layer.

The BART station has expanded over the period 1999-2016, and the BART maps reflect this. I have purposely excluded stations that did not exist in 1999 from the 2016 map. One station, Warm Springs, is due to open in the Fall of 2016 and has been excluded from all maps. Although track expansions after 1999 are present on the 1999 map, I have decided to include the full extent of the present-day BART track system for two reasons: 1) the tracks layer is not split up into expansions, which makes it impossible to drop the expansion portions without dropping the whole system 2) I felt that the tracks were illustrative of the expanse and direction of the present-day BART system, reinforcing my overall point about BART’s enormous growth.

As for the bubble maps, I decided to use equal interval sizes for the bubbles. This is mainly because it easier for the reader to understand. Furthermore, the data in percentage terms is fairly uniformly distributed while the exit maps in absolute numbers have large outliers, which would be lumped together with much smaller stations in some other distribution.

Finally, I decided to label stations because I mention some specific stations in the article. Also, some of my readers might be unfamiliar with the BART system. However, labels would clutter the map at first glance, so I have them only appear at a specified zoom level.