Report: “Multi-Modal Level of Service” Metrics Not Quite Up to the Challenge

1:26 PM PDT on October 17, 2014

UCLA researchers found that new Multi-Modal Level of Service metrics are not so great for measuring what's helpful for people walking and bicycling. Photo via Flickr user pranavbhatt — UCLA researchers found that Multi-Modal Level of Service methodologies are not so great for evaluating what's good for people walking and bicycling. Photo via Flickr user pranavbhatt

Livability proponents celebrate that car-centric Level of Service (LOS) is finally on its way out, at least in California.Wouldn't it be great if there was a similar bike- or walk-centric metric that could be used instead? UCLA Lewis Center and Institute for Transportation Studies researchers have studied some of the published metrics for evaluating how well streets serve pedestrians and cyclists. The researchers' conclusion: in all of the bike and ped metrics they reviewed, there is no silver bullet. Moreover, adapting LOS doesn't look like a fruitful approach.

Level of Service is a standard, simplistic engineering measure that uses car throughput to assign streets a letter grade from A to F. For nearly 60 years, LOS has been the predominant way that planners and engineers quantify streets. LOS guides policy and investment, justifying making streets wider and more car-centric. Because LOS only considers cars, LOS justifies widening roadways, adding turn lanes, and other so-called improvements that degrade conditions for people using bikes, their feet, or transit. To remedy this, transportation professionals have created new metrics designed to grade other modes. These include Pedestrian Level of Service (PLOS) and Bicycle Level of Service (BLOS), both of which are components of Multi-Modal Level of Service (MMLOS.)

There are some inherent problems with adapting a car-centric measure like LOS to bicycling and walking. Where people driving tend to see other cars as obstacles, pedestrians often prefer to walk where other people are out walking. Cyclists experience a safety in numbers effect, where LOS tends to just find congestion in numbers.

Though there are now many metrics from various parts of the world, UCLA researchers Madeline Brozen and Herbie Huff focused primarily on the following three metrics:

Multi-Modal Level of Service, included in the 2010 Highway Capacity Manual (HCM), the official federal evaluation tool developed by the Transportation Research Board of the National Academy of Sciences
Bicycle and Pedestrian Environmental Quality Index (BEQI and PEQI), developed by the San Francisco Department of Public Health
Urban Street Design Guidelines performance measures, developed by the city of Charlotte, North Carolina

Huff and Brozen used these tools to score sections of streets in the city of Santa Monica. They subsequently evaluated the sensitivity of the tools by comparing scores for existing condition to scores for potential improvements, including bike lanes, road diets, medians, etc.

In comparing existing sites to multiple potential improvements, the models showed "little variation," with none of the improvements appearing ideal or raising the score to an "A." Therefore, the models were ineffective for deciding between potential designs.

Perhaps the most troubling is MMLOS. This methodology is the most prominent; its published by the federal government as the way to evaluate bicycle and pedestrian Level of Service. It's billed as an "integrated multimodal approach to the analysis and evaluation of urban streets from the points of view of automobile drivers, transit passengers, bicyclists, and pedestrians."

Among the researchers' criticisms of MMLOS:

The Highway Capacity Manual's MMLOS is based on five studies, all from Florida, none with more than 150 participants, all from 1997 to 2005. None of the studies include facilities that were absent from Florida at this time: green pavement, protected bike lanes, etc.
In the PLOS formula, on-street parking is very favorably weighted, and it overwhelms the influence of other design factors. On-street parking is one of the main ways to improve the PLOS score, but conversely it reduces the BLOS score.
Due to an error in the BLOS formula, it is effectively "impossible to get a segment score of A or B."

Overall, the researchers assert that there is no simple, easy metric for transportation professionals to judge a complex multi-functional multi-modal street.

The study characterizes MMLOS thus--though this is arguably applicable to all these letter grade metrics (page 28 on this document [PDF]):

These single-outcome metrics can result in an overly simplistic, perhaps skewed understanding of how diverse user groups experience roadways in diverse ways. They [earlier MMLOS critics Kittleson and Roess] argue that LOS, to which we add the multi-modal metrics described here, offers the illusion of accuracy by providing precision. Analysts enter a number of inputs into a model and obtain a precise letter grade. But in doing so, these letter grades mask the variation in how different users perceive roads differently.

The results of the research are posted on the project website. They include:

A webinar recording and slide show [PDF]
A report containing an explanation of all measures, the comparison analysis, and the sensitivity case study: "Exploration and Implications of Multimodal Street Performance Metrics: What's a Passing Grade?" [PDF]
White paper containing a detailed breakdown of the variables and math used in both the bicycle and pedestrian calculations: "The Highway Capacity Manual's Method for Calculating Bicycle and Pedestrian Levels of Service: The Ultimate White Paper" [PDF].