I missed the controversy last January when Consumer Reports panned 10 of 12 infant car seats in their January issue – they retracted their safety alert 14 days later after manufacturers and government complaints caused CR to review their results and CR found the data flawed.
In their May issue (I get the paper copy - so I don't have a link), CR explains what went wrong with their testing. The root of the problem:
“…all the side-impact tests took place under conditions that could occur only if the striking vehicle were traveling at 70 mph or more”
This is a pretty significant error and I guess CR was too embarrassed to state it up front since one must read a page and a half of introduction before CR gets to the point. It’s easy to understand why the test occurred the way it did. CR explains their testing contractor misunderstood the side impact velocity requirement to be 38 mph after impact instead of before impact. To get the mass of two cars to move at 38 mph when one was stationary a split second prior to impact requires the traveling car to be moving over 70 mph. (see physics – momentum).
What CR doesn’t explain is how their engineers didn’t see the error once the data came in so far off the mark. I had a professor who would add additional penalties if a student didn’t acknowledge an obvious error for a test answer (sometimes you knew the answer was wrong but didn’t have enough time to go back and find the error). My fellow aircraft battle damage engineers called such a review a Sanity Check or the “Court Martial* Test” (i.e. would your assumptions pass muster in a court martial*).
CR’s error reveals that not much oversight was exercised here and perhaps they were more interested in making a news splash. It makes one wonder what might have slipped through in other tests. They took a big credibility hit on this; their May issue mea culpa is a good step in the direction to set things right.
Amazingly, two seats still passed - that has got to be quite an endorsement.
*Martial not Marshal - I thought it looked funny
















Comments (1)
Not the only instance of flawed methodology at CR - let's face it, they make money by highlighting differences, some of which aren't meaningful. For a better look at car ratings, for example, check out http://www.truedelta.com. He's as opinionated as anyone else, but he's working with real data.
Posted by greg | April 11, 2007 10:20 AM
Posted on April 11, 2007 10:20