Background
Observational audits are commonly used in public health research to collect data on built environment characteristics that affect health-related behaviors and outcomes, including physical activity and weight status.  However, implementing in-person field audits can be expensive if observations are needed over large or geographically dispersed areas or at multiple points in time. A reliable and more efficient method for observational audits could facilitate extendibility (i.e., expanded geographic and temporal scope) and lead to more standardized assessment that strengthens the ability to compare results across different regions and studies. Using omnidirectional imagery for observational auditing may support more efficient and extendable alternatives to field-based surveys but, given that this is an emerging technology, few studies have reported on the reliability of this approach. Omnidirectional imagery refers to the simultaneous collection of images in multiple directions from a single location, producing a panoramic view (exemplified by Google Street View). Previous research has found high agreement (mean prevalence-adjusted bias kappa, 0.81) between field audits and audits from Google Street View, indicating substantial to nearly perfect agreement among methods (Wilson, Kelly, et al 2012).

Objectives
To assess the inter-rater reliability of using Google Street View to conduct audits of the built environment.

Methods
Street segments from St. Louis and Indianapolis were geographically stratified to ensure representation of neighborhoods with different socioeconomic characteristics in both cities. Audits were conducted in 2011 using Google Street View. To assess inter-rater reliability, two researcher assistants in Indianapolis audited the same St. Louis streets, while two research assistants in St. Louis audited the same Indianapolis streets. Inter-rater reliability was assessed using observed agreement and the prevalence-adjusted bias-adjusted kappa statistic (PABAK). The PABAK statistic accounts for systematic differences between data sources and the distribution of each audit item (in other words, it accounts for low variability in the presence of some environmental characteristics).

Results
When measuring the inter-rater reliability, the PABAK statistic for land use characteristics was consistently above 0.80 (mean 0.87), indicating nearly perfect agreement in identifying residential and commercial destinations. The PABAK statistic for street characteristics was also consistently above 0.80 (mean 0.92). However, presence of a parking facility demonstrated lower agreement (mean 0.56), which is consistent with findings from previous research suggesting parking facilities are not as easy to identify using imagery as they are when conducting field audits. Street Characteristics (e.g. presence of sidewalks, continuity of sidewalks, presence of misalignments or obstructions) was also less consistent, with a mean PABAK of 0.69 (ranging from 0.39 for misalignments and 0.96 for presence of sidewalks).

Conclusions
The results of our analysis suggest that using omnidirectional imagery to audit the built environment is a reliable method for assessing majority of the characteristics included on the Active Neighborhood Checklist. Characteristics that demonstrated lower inter-rater reliability included condition of the sidewalks (e.g. sidewalk width and alignment) and presence of parking facilities.  Although auditors could easily identify if a sidewalk was present, it was diffıcult to determine the condition and width of the sidewalk from the imagery. Similarly, auditors could accurately identify parking lots and garages, but were less consistently able to identify if parking was allowed on street or not. A limitation of using imagery is that visual interpretation is limited to the camera’s perspective and vantage point, so auditors can not view the sidewalks or parking as closely as they would in the fıeld.

Reference
Wilson, J. S., Kelly, C. M., Schootman, M., Baker, E. A., Banerjee, A., Clennin, M., et al. (2012). Assessing the Built Environment Using Omnidirectional Imagery. American Journal of Preventive Medicine, 42(2), 193-199.

Support/Funding
Active Living Research