Let’s talk about tests. They can prove something works – or doesn’t work. They can identify good things in a heap of questionable things – or vice versa. A test can prove the feasibility of a cunning plan. Some tests are part of an exciting journey of discovery and others simply yield a pass or fail.
There can be unexpected outcomes. Failed tests fail to give a valid outcome; test failures succeed in giving a valid failure. Invalid tests fail to give a fail or a pass. Donald Rumsfeld must have something to say on this.
The politics of testing are complex. Bean-counters grudgingly provide funds so long as the test result will be a pass. Engineers regret having given in to the bean counters when they find themselves with a public failure. Regulators enjoy the occasional failure because it justifies their existence. Politicians would like to ban all testing and instead have a parallel universe in which to conduct control experiments and a time machine to remove all uncertainty.
So, when one reads “that a 2016 test of the UK’s submarine-borne strategic nuclear deterrent ended in failure” what does it mean? It could equally have been reported as “…ended having very successfully found a problem”. In any well-established industry tests are mandated following any complex series of changes where some form of end to end validation is essential. Such failures are universally met with a sigh of “thank heavens we found out”.
Navigation and guidance frequently feature in such scenarios. The complex mix of technology and people demands grown-up testing rather than the suck it and see voyage of discovery. A DNA print of a mature industry is its approach to validation and testing.
So where does the world of autonomous vehicles sit in the spectrum? “Mixed” is the answer. There are still many signs of immaturity. Public “look it works” demonstrations are not tests. Ironically, “look it doesn’t work” failures might give people a bit more confidence. Labelling all nay-sayers dinosaurs is childish. Claiming that all pioneers must take risks is neither true or relevant. Our pioneering forefathers would have died for the modelling and simulation tools we have now. Come to think of it – some of them did.
There are some good signs. I have just read a report by Atkins titled Research on the Impacts of Connected and Autonomous Vehicles (CAVs) on Traffic Flow. Stage 1: Evidence Review. Department for Transport. The grown-ups are arriving.
We’re heading the right way. We must avoid the logic which says we have no choice but to take risks, as in the much-cited scenario, “If the plane is going to crash anyway, we should storm the cockpit and see if one of us can fly.” It’s not how technical progress is made. We’ve yet to see if it works for politics.