“AI critical” ideas: a breakdown

Kombucha Girl Meme: Woman making an unhappy face at a stick figure saying “AI is bad”; Woman making an intrigued face at 4 stick figures each saying the same thing separately. In other words, “AI is bad” should be many separate ideas, not just one.

This story will be my attempt to organize and clarify my own “AI critical” thoughts. I’ll touch on various domains. In some of these domains, I have relevant expertise; in others, at least some decent degree of personal/anecdotal experience; and in others, I’m at best a thoughtful outsider, so laying out my thoughts will risk veering towards “mansplaining”. Furthermore, I know that others have wrestled with all of these issues, in many cases with expertise and insight beyond my own; while I’ll try to include references to such writing where I know about it, this will not be anything approaching a thorough or systematic literature review.

Still, I know that writing this out will help sharpen my own thinking, and I hope it will be useful for others, too.

In the following, I’ll use the term “AI”, even though I have reservations about whether it’s appropriate. In many cases, alternate terms such as “generative machine learning” might be better, but navigating this question over the various issues I’ll discuss would be a distraction.

Of course: this article is about anti-AI critiques, but should not be taken to imply that AI (either in general, or in its current incarnations) doesn’t have it’s strengths and valid uses.

Issues with “AI”:

1. Current models fall short of “intelligence”; and this is a structural/systematic problem, not likely to be solved by incremental/imminent improvements. (“Artificial stupidity”)
2. AI tends to learn, and thus reproduce, existing biases. For instance, AI’s are often bigoted in various ways that are not easy to avoid or solve.
3. AI as a tool is available to those with resources, and thus tends to widen existing inequalities in wealth and power.
4. AI training data is often collected without consent, and thus AI is often a tool for (quasi-)plagiarism.
5. Using AI may reinforce laziness and/or reduce chances for the user to learn new skills. This is especially the case when it’s used for schoolwork or similar cases (eg, apprentices/interns for “training” tasks).
6. AI may enable bad actors and thus help spread scams, misinformation, and the like.
7. AI uses resources — especially energy, and even more especially fossil-sourced energy—that could better be spent elsewhere or never used at all.
8. Current AI “achievements” often actually rely on substantial amounts of human labor, and the current conditions for that labor are generally exploitative / not sustainable.
9. At some point in the future, AI may become self-improving and/or superhuman in important ways. At that point, it could pose superhuman dangers, including bringing about a future we’d consider abbhorent (eg, exterminating or enslaving humanity; “GAI existential risk”).

Note that, while each of the above could, in principle, exist independently of the others, they can also interact and reinforce each other. For instance, problems #1 and #2 combine to more than either alone, and similarly with #4 and #5, or #4 and #6, or #7 and pretty much any of the others.

However, #1 and #8 are in tension. It’s my personal opinion that #1 is more of a factor in the near future — let’s say, the next 20 years or so. IMO #8 is a valid concern over the longer term, but is far overblown by those who think it’s an immediate issue, in a way that actually worsens some of the other problems. I think that almost all of the effort to address #8 is best spent directly on #3: addressing inequalities as they relate to AI.

Over the coming days, I’ll continue to edit this article, adding longer explanations below for each of the issues; but I think even just writing out this list is useful, so I’ll hit “publish” now.

Digging deeper

I’ll try to explore each of the above points separately below. In doing so, I’ll look at how they vary along different spectra:

• How serious a problem is this?
• How sure am I of that?
• Is this more of a fundamental problem with AI, or is it relatively superficial/fixable/circumstantial?

1. Current AI models aren’t really “intelligent”

Current AI progress is dominated by generative models trained by large-scale, parallelized, gradient descent machine learning. This means a paradigm generally based on (most of) the following steps:

• Assemble a large corpus of training data.
• Build a fixed architecture, generative “neural net” (basically, a series of parametrized tensors and simple nonlinear operators of a limited depth), with affordances for projecting training data points into a “conceptual vector space”, for seeing relationships in that conceptual space across space or time.
• Train that generative neural net using parallelized gradient descent — that is, adjust parameters using calculus to minimize a loss function. Early training generally sets the loss function to be something like “reproduce the training data”; later stages might involve fine-tuning using different loss functions, such as RLHF (reinforcement learning from human feedback).
• Eventually, stop using gradient descent, and just use fixed parameters to generate output.

Note that not all modern AI uses all the steps above. For instance, AlphaZero and its descendants use self-play to generate its own training data, and its loss function is, from the start, based on some idea of “winning” rather than just imitating the training data.

Nevertheless, while the above paradigm has led to impressive advances, I believe it differs from “true intelligence” (as exhibited by humans at least some of the time) in several key ways; and that these issues are fundamental enough that it’s unlikely that solutions are just “right around the corner”.

One key difference is that, in this paradigm, “learning” and “thinking” are completely separate. “Learning” is done via back-prop and gradient descent, ahead of time; “thinking” leaves that out entirely, using pretrained parameters in a purely forward sense. And, while I’m sure there are specific modern protocols that begin to reduce this hard barrier, the fact is that it’s to a large degree fundamental to the entire gradient descent paradigm which, I’d argue, is behind essentially all modern AI breakthroughs.

Let’s call this separation into largely-separate “learning” and “thinking” modes “L/T separation”. Is it incompatible with true intelligence? Perhaps not; after all, human brains also run in at least two largely-separate modes — “asleep” and “awake”. While both of those modes seem to involve some degree of both “learning” and “thinking”, the example still should give pause before assuming that intelligence must be constructed through a single holistic process.

Still, the L/T separation does seem at least deeply alien to how human intelligence works, and arguably problematic. It seems introspectively obvious that, when facing a difficult problem, often the best solutions come from a trial-and-error process in which learning and thinking are inextricably entwined. It’s not per se impossible that AI might find some adequate substitute for such an entangled process, but it certainly makes it seem a higher hill that AI must climb.

Furthermore, the fact that L/T separation is so foundational to modern AI, leads to several further implications in terms of what kinds of structures work best built on top of that foundation.

• Efficient training rounds on modern GPU chips require massive parallelism. That pushes towards a “wide, not deep” model architecture which arguably differs from key aspects of intelligent thought; and while things like transformer architecture use attention heads to effectively “deepen” things, I think they still do so in sharply limited ways.
• The exigencies of parallelized training rounds, as well as coding and design practices, push structures to be relatively uniform, and with minimal branching logic.
• Backprop along “wide, not deep” structures with massive numbers of parameters produces dense gradients, requiring a low learning rate so that learning is cumulative rather than having each new learning round simply wipe over the previous ones. This, in turn, accentuates the insatiable hunger for training data, and leads to an inhuman learning style that’s inherently unsuited for real-time learning. (Though advances in “few-shot” learning might superficially seem to invalidate this critique, many such advances actually exist only within the “thinking” branch of the L/T separation, thus don’t really address the problem.)

Then, there are the issues with gradient descent itself. However you define intellegence, one clear requirement is the ability for goal-directed behavior; and gradient descent is a convenient mathematical trick that directs behavior towards minimizing the loss function. But I’d argue that true intelligence requires the ability to learn and change goals themselves, not just pursue them. Meanwhile, nearly all modern AI gradient descent is done using a loss function that’s pre-fixed relative to some external metric, such as “winning the game”, “reproducing the training data”, or “reinforcing towards/away from behaviours the human feeder-backers like/dislike”.

Which brings us to my last, and in some ways strongest, critique of “intellegence” (as applied to current ML models): tuning a network to reproduce the training data—to imitate intelligent behavior—will never lead to actual intelligence.

I’m certainly far from the first to have this critique. In Van Rooij et al 2024, the authors actually give a rigorous proof that learning a broad behavior pattern such as “human intelligence”, purely by “imitation”, is effectively impossible, even under some incredibly-favorable assumptions about sampling conditions and computing power. It’s true that there might be a chink in their logic, in that one might assume that “human intelligence” is not just some arbitrary function, but has some favorable characteristics that make it particularly easy to learn. But at least for me, it seems intuitively highly likely that they’ve successfully shown that current AI modes of research will not and in fact can not lead to true intelligence*, at least given all the other critiques I’ve given above.

* Internal footnote: They go further, claiming that they have shown that *no* modes of research can lead to artificial intelligence. But to me this seems obviously too far; by the same logic, evolution could not have led to intelligence. What I think they have shown is that you can’t get intelligence simply by trying to imitate existing intelligence; that you need to learn from action, not imitation. I may be wrong, but it seems to me the AI field, diverse as it is, is not close to breakthroughs in that regard; that current breakthroughs, while they are real and useful, are only a mirage in terms of seeming to get closer to general AI.

One basis for that intuition of mine comes from the game of Go. I’m at best a mediocre Go player — my ranking seems to fluctuate between 3 and 7 kyu, where any credible/serious player has a rating “below” 1 kyu. But it’s certainly my experience that it’s incredibly difficult to learn better play simply by studying games of those substantially better than you are. For every stone that’s placed in high-level play, there are dozens of alternate possibilities which the player has discounted as leading to worse outcomes, and those exponentially-many never-explored failures are what I as a beginner need to understand to improve my play, not the narrow path of expert (near-)success. I think that intelligent human behavior is similar: to understand why somebody does something, you need theories of why they didn’t do countless alternatives; and, at least given all the limitations I’ve listed regarding current AI models, I think they’ll remain eternally several steps behind as long as mere imitation is such a key part of how they work.

But, wait a minute, you might be saying. How can I use Go as an example of why computer intelligence is “impossible” using current paradigms, if in fact AI-based Go programs now easily outplay the best humans?

Well, note that the best computer Go programs, beginning with AlphaZero, do not proceed via imitation; they play countless games against themself, learning from what works and doesn’t. Could some similar trick lead to true general AI (GAI)? In the long run, and for sufficiently-broad definitions of “similar”, I think that of course it could. But in the short run, I think not. As I’ve mentioned already, gradient descent loss functions are generally pre-written, and refer to objective external criteria like “who won, according to the rules of Go”. But human action involves dynamically and internally setting up sub-goals and heuristics, in a way that I haven’t seen any current AI really reproduce in a scalable way. I think that’s actually part of why imitation is a central pillar of modern AI: it provides apparent progress towards such dynamic-goal-based behavior… but only in a way that’s ultimately undermined by Van Rooij et al’s critique.

…

Above, I’ve given roughly six more-or-less-distinct critiques of modern “generative gradient descent”-based AI paradigms. I want to clarify that I am NOT at all arguing that machine intelligence is impossible in theory or practice. I believe that all the important processes happening in our brain (and glands etc.) to produce (at least sometimes) intelligent behavior, are conventional computations. (Quantum computers are interesting but IMO irrelevant here; I could say a lot more on that but for this essay I’ll ask you just take it as an axiom). I believe that eventually, humans will produce some true GAI. It’s just I don’t think current work is anywhere close to that threshold. I’d bet that we won’t see GAI in, say, the next 20 years.

How sure am I of that? After all, if I had one good solid reason, I probably wouldn’t need six. Each of my six arguments rests on at least one sweeping generalization to which there are exceptions I already know of, and probably many more that I don’t. And there are many people who are far smarter and/or more familiar with current cutting-edge AI work than I am, who would, with pretty high certainty, take the other side of the 20-year bet I offer above (for some specific definition of “GAI” that we could probably agree on).

Still, I expect that in retrospect, 20 years from now, around three or four of my six arguments above, including the last one, will seem to have been generally on-target. So while I’m not strongly attached to each of the arguments above individually (and indeed, I could lay out further arguments contra and pro on each), I’m relatively confident that my conclusion is sound. (On a meta level, I understand it’s possible I’m just stupid and the people on the other side of this are right; but if so, that stupidity would pollute any of my introspection about my own certainty, so only such non-probabilistic external meta heuristics would help me avoid overconfidence.)

2. AI tends to learn, and thus reproduce, existing biases.

For this conversation, I’ll break down biases like racism, sexism, etc. into three subtypes:

• Hard bigotry: the belief that a certain group of people are inherently inferior in their capabilities and/or human value.
• Soft bigotry: seeing existing disparities in group outcomes and acting in a way that accommodates and ultimately accentuates them rather than fighting them, without hard bigotry behind it. (This is, essentially, what happens when you know that correlation isn’t causation, but use it as a proxy anyway.)
• Structural bias: building systems using cultural/material assumptions that are based around a dominant group, and thereby unintentionally creating bias against other groups.

AI training data, and thus “raw” AIs such as LLMs, will naturally contain all three subtypes. And I’d argue that, while it may be possible to use RLHF to at least minimize hard bigotry, AI is especially subject to problems with soft bigotry and structural bias.

To understand why, go back and look at the architectural critiques in point 1 above. Generally speaking, these critiques suggest no impediment to AI capably learning even very subtle/involved correlational structures of its training data. Instead, they suggest that it will be difficult or impossible to get AI to take a truly critical / agentic stance towards that data; for instance, for it to “care” about the difference between correlation and causation (as necessary to avoid soft bigotry), or about accommodating minority needs (as necessary to avoid structural bias).

In linking this to point 1, my intention is to suggest that this is not at all a superficial or easy-to-fix problem. I believe that using AI without serious caution in this regard will not just perpetuate existing biases, but in many cases worsen them. I don’t doubt that there are contexts and ways to use AI carefully that might avoid this, but such care must be integrated and serious, not just an afterthought.

3. AI as a tool is available to those with resources, and thus tends to widen existing inequalities in wealth and power.

I don’t have a lot to say about this one. In my personal opinion, this would only worsen existing inequalities insofar as AI is uniformly more effective than alternate ways of solving the same problems. But, because of all the other issues I discuss here, I don’t think that’s particularly the case (yet, or even imminently). So I think that yes, it’s important to fight inequality overall, but that current AI doesn’t make that much more necessary than it already was. (Note that “current” is key to the preceding sentence; I’ll come back to this point when I discuss existential risk.)

4. AI training data is often collected without consent, and thus AI is often just a tool for (quasi-)plagiarism.

This is true, and reasonably fundamental to the way the AI industry generally works. I believe that, for this reason, it is simply unethical to use AI to replace and/or de-skill creative workers: to produce art, illustrations, screenplays, ersatz actors, etc. However, there are a lot of uses of AI that are not along these lines—perhaps even including use within a creative process as directed by an artist with care for their professional skills that are being augmented rather than replaced—and I’m not convinced this issue makes all of those other uses problematic.

5. Using AI may reinforce laziness and/or reduce chances for the user to learn new skills. This is especially the case when it’s used for schoolwork or similar cases (eg, apprentices/interns for “training” tasks).

A large percentage of the public use of LLM-based chatbots (ChatGPT, etc). is by students for schoolwork; as evidenced by, for instance, substantial drops in use during school vacations. While there may be ways to use ChatGPT to enhance learning (perhaps as a Socratic partner?), it seems clear that “do my homework for me” is not one of those.

6. AI may enable bad actors and thus help spread scams, misinformation, and the like.

AI has certainly been used by scammers and propagandists to create credible lies in text, voice, images, and video. There is various work on fighting this, but it doesn’t seem that the “white hats’” progress outpaces that of “black hats”.

7. AI uses resources — especially energy, and even more especially fossil-sourced energy — that could better be spent elsewhere or never used at all.

8. At some point in the future, AI may become self-improving and/or superhuman in important ways. At that point, it could pose superhuman dangers, including bringing about a future we’d consider abbhorent (eg, exterminating or enslaving humanity; “GAI existential risk”).

Bibliography

Van Rooij, Iris, Olivia Guest, Federico Adolfi, Ronald De Haan, Antonina Kolokolova, and Patricia Rich. “Reclaiming AI as a Theoretical Tool for Cognitive Science.” Computational Brain & Behavior, September 27, 2024. https://doi.org/10.1007/s42113-024-00217-5.

…

Here’s another version of the first image above, in a more medium-friendly aspect ratio:

Same Kombucha girl meme above, first frame only (for teaser).