by Joseph P. Farrell, Giza Death Star:
Many people have been trying to warn of the dangers of artificial intelligence, my own small voice among them. And now, there is a strange article from Zero Hedge that draws attention to the dangers in connection with those Boeing 737 Max 8 airplanes, two of which have crashed – with fatalities – in recent months, prompting several airlines either to ground their fleets, or to cancel orders. Mr. E.G. and Ms. K.M. spotted this article, and as one might expect, it prompts some high octane speculation, or rather, perhaps a revisit to some of my earlier warnings about the increasing reliance on AI:
The problem, according to the article, is an “MCAS patch”, a safety anti-stall program on the aircraft, and it’s worth noting what this article says about it:
I think the problem is that the Boeing anti-stall patch MCAS is poorly configured for pilot use: it is not intuitive, and opaque in its consequences.
By the way of full disclosure, I have held my opinion since the first Lion Air crash in October, and ran it past a test pilot who, while not responsible for a single word here, did not argue against it. He suggested that MCAS characteristics should have been in a special directive and drawn to the attention of pilots.
And there’s another problem:
Boeing had a problem with fitting larger and heavier engines to their tried and trusted 737 configuration, meaning that the engines had to be higher on the wing and a little forwards, and that made the 737 Max have different performance characteristics, which in turn led to the need for an anti-stall patch to be put into the control systems.
So the patch was put into the system. But then there’s this at the end of the article:
After Lion Air I believed that pilots had been warned about the system, but had not paid sufficient attention to its admittedly complicated characteristics, but now it is claimed that the system was not in the training manual anyway. It was deemed a safety system that pilots did not need to know about.
This farrago has an unintended consequence, in that it may be a warning about artificial intelligence. Boeing may have rated the correction factor as too simple to merit human attention, something required mainly to correct a small difference in pitch characteristics unlikely to be encountered in most commercial flying, which is kept as smooth as possible for passenger comfort.
It would be terrible if an apparently small change in automated safety systems designed to avoid a stall turned out have given us a rogue plane, killing us to make us safe.
I don’t know about you, but I find all this profoundly disturbing, because what is being alleged in the article is something like the following:
1) The MCAS “patch” was “too opaque” for humans; it was “not intuitive”;
2) It was therefore kept out of the pilots’ training manuals because of this; and
3) The cumulative effect of these decisions means pilots were essentially not in control of their aircraft, and the system itself may have crashed them
Most people are aware that I don’t fly at all. I used to, but even then I was never comfortable doing so. These days, I just simply flatly refuse, and in part it’s because of stories like this. And we’ve already seen similar stories with “self-driving cars”. Automated semi-trucks are reportedly already on the road, though we’ve yet to hear of any accidents involving them or failures of their AI. But whether or not it’s true that automated AI trucks are driving down the road, I can say that there’s something else that has happened. Not too far from where I live, the local Burlington Northern Santa Fe railroad does run freight trains with no engineer in the cab of the locomotives. There’s even a little sign to this effect at a prominent railroad crossing that I often cross, or rather, used to cross until the impact of the sign warning about the automated train finally sank in. Now I drive a block or two out of my way to cross over the tracks on a bridge.