Energy End-Use Forecasting
EUF > Project Pages > Evaluation
Evaluation of I/M Program Effectiveness

Project Description

We have done an extensive evaluation of the Arizona IM240 program, using both program test results and remote sensing data. Analysis of IM240 test results from Arizona confirms EPA's findings ( http://www.epa.gov/orcdizux/regs/im/az-rpt/az-rpt.htm) regarding the program's overall effectiveness (Glover and Brzezinski. 1997. Analysis of the Arizona IM240 Test Program and Comparison with the TECH5 Model. May.) The initial reduction in fleet emissions due to repair compares well with TECH5 predictions for CO and HC, on the order of 15%, while the initial reduction for NOx is only half that predicted by TECH5 (7% as opposed to the predicted 17%). However, our research found that the effectiveness of I/M programs is affected by:

  • program avoidance: One-third of all vehicles that fail initial I/M testing never receive a passing test; About one-third of these vehicles are observed by remote sensors still being driven in the I/M area more than 2 years after their last (failing) I/M test. In addition, 40% of all vehicles tested in the first year of Enhanced program in Arizona (1995) were not tested in the next cycle (1997).
  • ineffective repair: 40% of vehicles that failed their initial test in 1995 failed again in 1997. About half of these failed for the same combination of pollutants in both years. Remote sensing data indicate that repair effectiveness drops as soon as a few months after a vehicle's final I/M test.
  • pre-test repairs: The remote sensing data also indicate that average emissions decrease substantially about three weeks prior to the initial I/M test, presumably due to pre-test repairs and/or adjustments (see Figure 1).

Figure 1: Average CO RSD Emissions by Time Period

Analysis of vehicles reporting for two I/M cycles reveals that the effect of two years of emissions deterioration outweigh the effect of the I/M program; the after program emissions in 1997 are substantially higher than the after program emissions in 1995 (Figure 2 shows initial and final IM240 test emissions in tons per day in 1995 and 1997). In addition, there are large fluctuations in the vehicle fleet tested in each I/M cycle: 40% of the vehicles tested in 1995 do not report for testing in 1997, while 40% of the vehicles tested in 1997 were not tested in 1995.

Figure 2: Fleet Emissions Over Two I/M Cycles

These results are summarized in a July 27, 1999 memo (33K.pdf) to EPA, and have been presented at the NCVECS Clean Air Conference in 1998 and 1999 (455K.pdf), to the NRC Committee to evaluate MOBILE (618K.pdf) in March 1999, and at the 1999 CRC Workshop. (520K.pdf) These data were also used to prepare comments to EPA on MOBILE6 (130K.pdf)We recently presented lessons learned from our I/M evaluation activities (683K.pdf) to a NRC Committee on Effectiveness of Vehicle Inspection and Maintenance Programs.

We also have examined program effectiveness for all vehicles subject to I/M testing in the Phoenix area, including older vehicles subject to loaded idle testing. These results are summarized in a report for the Arizona Department of Environmental Quality (358K.pdf).

We provided an in-depth evaluation of California's Enhanced Smog Check Program, Smog Check II, for the state Inspection and Maintenance Review Committee. The evaluation made use of millions of I/M test records, 30,000 roadside tests of randomly selected vehicles, 150,000 remote sensing measurements, and nearly 50 million vehicle registration records. Because California requires additional I/M testing when vehicle ownership is changed, we were able to analyze multiple cycles of I/M test results in a 12-month time period on a portion of the vehicle fleet. Our analysis of the California multi-cycle vehicle fleet found that:

  • 20% of vehicles failing their initial I/M test and passing a retest ("fail-pass" vehicles), and 6% of vehicles passing their initial I/M test ("initial pass vehicles"), failed a subsequent initial test one month later. These vehicles failed so soon after passing an I/M test either because of inherent test-to-test variability in vehicle emissions, or fraudulent test practices.
  • The failure rate and average emissions of the fail-pass fleet remain fairly constant over the next 9 to 12 months, indicating that any repairs made to these vehicles are effective and durable.
  • The failure rate and average emissions of the initial pass fleet increase dramatically over the next 12 months, with the failure rate nearly tripling
  • Comparison of initial to final I/M test results over a single cycle of I/M testing overstates emission reductions. A more accurate estimate of emission reductions involved comparing initial to initial I/M test results over subsequent I/M cycles.

We also analyzed the degree of program avoidance, the effect of pretest repairs and maintenance, repair effectiveness by station type, and other program elements. We estimated the total tons per day exhaust emissions benefit of the Enhanced program, by source of emission reduction and vehicle model year. The IMRC approved a report of our findings at their June 19th meeting.

The full report, including appendices, can be downloaded at http://www.smogcheck.ca.gov/smogweb/IMRC/

We provided a peer review of an I/M evaluation methodology proposed by Sierra Research. We also have been working with a group of experts to help EPA write guidance on how to best use remote sensing data to evaluate I/M programs (forthcoming).

Project Staff

Key Data

Publications

Other Resources

Energy Star energy conservation energy savings energy efficiency environment climate change weatherization solar energy appliances energy audit energy tools efficient buildings energy education heating cooling heat pumps furnaces boilers air conditioners insulation ducts skylights windows thermostats fuel switching mechanical ventilation water heater water heating refrigerator freezer dishwasher clotheswasher clothes dryer miscellaneous energy home energy comfort remodeling residential energy.

TEMA Group Site The Enduse Forecasting Group, Lawrence Berkeley National Laboratory
 Last Updated On: 8/19/04