Current AIs are a lot like traditional Chinese medicine, says Saffron Huang. We value their results, but the science behind them is not yet clear
From New Statesman
A fascinating piece from Wired magazine. It’s Saffron Huang, a researcher into large-language-model AIs, who has a strong background (from her parents) in Chinese traditional medicine, particularly acupuncture. And she sees intriguing synergies in how both of them function. Some excerpts:
Now that I work in machine learning (ML), I’m often struck by the parallels between this cutting-edge technology and the ancient practice of traditional Chinese medicine. For one, I can’t quite explain either satisfactorily…
Acupuncture works, but we aren’t sure why. Langevin and Wayne, both Harvard Medical School researchers, have suggested that although acupuncture has become more empirically legitimized, it is held back by the theory behind it. The idea of qi flow as the essential variable, a body-society whose health depends on the state of its networks, is an elegant but inadequate metaphor…
…There’s little evidence so far for the theory underpinning acupuncture, but there is decent empirical evidence for acupuncture itself. This is surprisingly similar to AI. We don’t really understand it, the theory is slim and unsatisfying, but it indisputably “works” in many ways.
…In large neural networks like GPT-3 or Bloom, there are hundreds of billions of numerical parameters. From this, what can we deduce of the actual mechanics of the rule-set unearthed by the iterative learning loop, the reasoning steps by which the perfected neural network performs its logic? Like the causal pathways through which Chinese medicine might exert its reason, we have no idea.
In statistics, there is a known distinction between explainability and prediction, as well as a trade-off: The most accurate explanatory model of a phenomenon is not always the best predictive model. Machine learning is an approach that accepts the predictive power of the Faustian bargain, trading away explainability.
Following a surprisingly similar framework, the herbal or acupuncture prescriptions that my father gives me are based on a holistic evaluation of my personal and environmental data points (e.g. seasonal changes in the weather, my diet, my stress levels), information that I’ve not seen conventional doctors ask for.
The prescriptive outputs are also strangely specific yet mysterious in origin, just like language model generations. Dad will prescribe me mixtures of 10 or 15 different herbs, and I’ll ask him how he came up with the formulation, but I won’t understand the explanation—something about qi interacting in various parts of my body.
My poor comprehension might be attributed to my limited grasp of traditional Chinese medicine, or the fact that this system, like machine-learning, isn’t built for easy mechanistic understanding. Then again, the explanation may not actually correspond to whatever mechanisms the herbs act through...
The field of AI used to have easier-to-interpret algorithms, ones that don’t depend on inscrutably entangled learnings. People used to start with logical building blocks to put together more complex systems, assuming humans and machines can reason by following a set of formal rules that specify what they should do in any circumstance.
This is now known as “good old-fashioned AI”; by and large, researchers have abandoned this in favor of computational systems that can, through trial and error, cybernetically correct themselves toward the best outcome. We embrace the messy, emergent logic that comes out of this process, rather than building up a modular logic, because we’re won over by the great predictive power of this approach.
We now try to squeeze explanations out of the more complex model, hoping that we can find some understandable structure within. But there’s no reason to expect a simple logic to fall out, and current approaches to “explainable AI” aren’t guaranteed to correspond to the real internal “reasoning” of the model. Much like the current theories of acupuncture that Langevin and Wayne criticized, a false theory is worse than no theory.