Alan Turing, the father of theoretical computer science, made a proposal that has put AI development on a flawed path for three-quarters of a century, a prominent computer scientist has argued.

In his provocative new analysis in Turing’s Mistake: Escaping the Yoke of Unintelligent Machines, Peter J. Denning tells us that Turing’s stance in 1950 reflected a belief held by the scientific community at the time: that human intelligence can exist without a body – and can therefore emerge in software on digital computers.

Denning also challenges the belief that machine intelligence can be confirmed through an imitation game (now known as the Turing test).

“These two claims have shaped much of AI research and development,” Denning writes. “My premise is that our acquiescence to these claims has led to the AI mess in which we find ourselves today.”

He argues that the artificial intelligence (AI) society is headed for won’t yield a human-level intelligence, known as artificial general intelligence (AGI) – instead it will be dangerous, he warns.

The tacit knowledge problem

At the core of Denning’s argument lies a concept called tacit knowledge. Tacit knowledge represents the vast area of human understanding that cannot be articulated in words or captured in any symbolic form that machines can process.

Denning identifies five major domains of tacit knowledge that he says ‘elude machine learning’. These include common sense knowledge, our daily interactions with others and our environment, our feelings and perceptions, performance skills, and our social and historical culture.

Humans have tried to catalogue common sense knowledge. Beginning in the 1980s, Douglas Lenat’s ambitious Cyc project attempted to compile a comprehensive database of common-sense facts. After 40 years of human effort, it had accumulated 25 million entries.

“Yet even this treasury could not add up to a background of common sense sufficient to make expert systems smart enough to be experts,” Denning notes. “Cyc validated that much of the knowledge that makes people experts cannot be articulated as propositions.”

Performance skill presents another insurmountable barrier.

“Our performance skills in thousands of domains cannot be communicated to machines,” Denning explains. “Whereas descriptions of skillful outcomes (‘know what’) can often be represented as bits and stored in a machine, we do not know how to encode the embodied knowledge for skillful performance (‘know how’).”

Musicians demonstrate this this gap. Denning says: “A virtuoso violinist can play beautiful music yet cannot describe to an acolyte how to produce it.

“Even if a robot could observe and imitate skilled humans, having no biological body, a robot cannot grasp how the musician feels when playing beautiful music or how an audience feels when hearing it.”

Other examples of tacit knowledge include intuitions, gut feelings, spontaneous creativity, and imagination.

The unbreachable barrier

The barrier to all of this is what Denning identifies as ‘the representation problem’.

This fundamental obstacle to achieving human-level AGI is because for any computation to occur, data and instructions must be encoded in physical forms that machines can recognise and process. But tacit knowledge, by its very nature, resists such encoding.

“Behind every word is a deep well of tacit knowledge that gives it meaning,” Denning says. “Words are but symbolic representations of meanings, not the meanings themselves. Commonly used Large Language Models, such as ChatGPT, Claude and Gemini only manipulate words, they cannot know or understand the meaning of what they are saying.”

This creates an unbridgeable divide – because we cannot explain or even understand how tacit knowledge works for humans, we cannot begin to communicate it.

“How we host tacit knowledge is largely a mystery,” Denning admits. “All we know is that it is embodied. We have no idea what we might observe and measure in our bodies to reveal it.”

Context and culture

Beyond individual knowledge, Denning emphasises the role of context – or the circumstances of a situation which gives our statements and actions a broader sense of meaning and purpose.

Context provides innumerable layers of meanings that extend beyond any horizon. Context provides the clue to whether someone is being sarcastic or sincere, or if someone is angry or teasing. Context tells us whether to employ tact or use humour.

“When you inquire into where an assumption of the current context came from, you discover it rests on previous conversations from previous contexts. Each of those in turn rests on further previous conversations and their contexts. This pattern is endless and fractal,” Denning explains.

The cultural dimension of intelligence poses similar challenges.

Culture encompasses our values, norms, judgements, histories, communities and moods, even dynamics of power or care.

“Human conversations are imbued with background assumptions that give meaning and relevance to the words being used,” Denning explains.

“Scaling up LLMs with ever larger neural networks will not enable them to acquire the embodied human knowledge we call culture. LLMs will not attain the objective of the Turing test: to demonstrate machine thought indistinguishable from human thought.”

Ultimately, Denning says there is a mutual incomprehension between humans and machines:  artificial neural networks will create a form of machine tacit knowledge that humans cannot understand.

“Machines cannot read our tacit knowledge and we cannot read theirs,” he writes. “We are aliens across an uncrossable divide.”

This has profound implications for AI safety. As machines are unable to read unarticulated human context, aligning them reliably with our intentions may be impossible, Denning warns.

“Through AI automation, agentic networks of machines are likely to develop their own machine intelligence that does not reach the level of human general intelligence but is still quite capable of creating severe problems for humans. This threat is a greater than a take-over by superintelligent machines,” he explains.

“Machine intelligence has different concerns from us and does not appear to care about us. Its ways of thinking and problem-solving look alien to us. We do not yet know how to live safely with these machines.

“Pulling back from an AI automation singularity will demand much from us. We start by accepting that the familiar culture is fading away as intelligent machines appear in our society and we do not know what is coming. We decline to think like machines or be subservient to machines. We refuse to submit to a yoke imposed by low-intelligence machines. Most importantly, we reassert our humanity, declare once again what makes us different from machines, and celebrate those differences.”