Oh god, OK—yes, I’m sorry, more “#AI” discourse, and believe me, no one finds my repeated returns to the topic more wearing and ironic than I do, but we are living in A Moment, and I have been wrestling with that moment on the personal and artistic front, but I have also been looking at it professionally—and still am, because that’s the question people want to discuss!—and that means I Have Opinions about both the technology and the discourse around it, and when I see smart people doing dumb discourse I just can’t help myself, because I came up as a blogger and that’s just how this writing-and-thinking game works for me, OK?
So, yeah: Philosophy Bear has a rant titled “You’re not going to stop AI by pretending to be unimpressed by it”, prompted by something someone said on one of the “dirtbag left” podcasts… and really I could probably end this critique here by saying “dude, stop listening to that shit; it’s just going to wind you up, that’s the whole point”.
But they didn’t leave it, and so I’m not going to leave it either.
Let’s get stuck in, then: the Bear has a set of beefs with the “autocomplete on steroids” analogy, and I’ll respond to them in order. That analogy:
A) Is a misleading description of the model once it has undergone reinforcement learning through human feedback.
I was under the impression that autocomplete is also reinforced by feedback, but I’m not an expert, so I’ll concede this one.
B) Completely misses the key insight behind this era of LLMs. Predicting the next word can be used, with minimal setup, to reproduce useful human behaviors like correctly answering questions (even difficult reasoning questions) and producing requested text.
I don’t think this insight is missed, but the conclusion which (for the Bear at least) implicitly follows it may well be missed; it certainly is missed for me.
OK, sure, these systems can answer questions and produce text on demand. What is that good for, exactly? Which jobs can this actually viably replace? I’ve started asking the more breathless boosters on LinkedIn this question in recent weeks: tell me one job it will replace, give me one “killer application” of generative systems.
I never get a reply. Not even a machine-generated one.
C) Gives people a false confidence that they understand what is going on inside the model. The fact that something is a model of the most likely next word does not negate the possibility that it has, within its parameters, also a model of how the world works. Past a certain point, understanding the world in at least a functional sense is necessary for predicting the next word.
I have zero confidence that I understand what’s happening inside the model. I’m given to understand that most folk who actually work on the comp-sci end of things readily admit that they don’t understand what’s happening inside the model. As such—and with respect—I’m pretty skeptical that a philosopher of technology understands what’s happening inside the model any better than either of us.
Which is why claims about “a model of how the world works” seem crazy to me. Firstly because that’s a model of a model of a model, to stay within the analytic tradition’s framing. But secondly because, for those of us on Team Continental, the very most that a large language model can be is a model of language-as-used, and while language might be considered to be a map of the territory that we might label “how the world works”, it is not a very good map, and a deeply contested one in constant flux.
Which brings us to:
D) Won’t be very comforting when everyone whose job can be done by receiving text and responding with text has been replaced with “““autocomplete on steroids”’”
Ugh.
I don’t listen to Chapo, but I do listen to Cory Doctorow, who may not be a philosopher, but who has forgotten more about technology than will ever be learned by me and the Bear put together, and—perhaps more importantly—knows the discourses of the Silval scene from both sides of the fence, poacher and gamekeeper. Cory’s line is “AI isn’t going to do your job, but its narrative may convince your boss to fire you and replace you with a bot that can’t do your job”, and I find that a convincing argument in the present, but also historically. It is no coincidence that Luddite Discourse is making a comeback right now! Generative systems are not paperclip-maximisers in waiting; they are weaving machines.
That said, a freshly minted philosophy PhD worrying about their employment prospects is both forgivable and relatable. I mean, philosophy is about as pure a language-in-language-out game as you could ask for, right? A game of “correctly answering questions (even difficult reasoning questions) and producing requested text”?
The sudden discovery of LLM material just sat there openly in academic journal papers which were supposedly subjected to peer review is a grim harbinger, certainly—but in the presence of the structural (dis)incentives that apply to academia in the present moment, it should come as no surprise, really. The prevailing model of student assessment is also fucked, but that model developed in response to an environment in which universities were rewarded for becoming engines of crank-’em-through credentialism. If it wasn’t LLMs fucking things up, it would be something else equally shoddy and “disruptive”, because that’s the only thing that attracts investment nowadays.
Which is why I’m almost amused by the recourse to a “line go up” argument by the Bear:
What numbers must I show you to make you take this seriously?
Ah, my dude—the very middle of the S-curve always looks like an exponential when you’re stood there looking upward.
Moore’s Law was never a law, it was a sales pitch—and an incredibly successful one, whose seductions are still working today! “Multi-Task Langauage Understanding”, meanwhile, is IQ but for machines—and as we should all know by now, the only thing that IQ measures is how good you are at taking IQ tests.
These firms are not just marking their own homework, they’re setting the questions, too. That they’re repeatedly coming home with a straight-A report card should be seen as suspicious rather than threatening.
But the killer weakness of the Bear’s argument here is not philosophical but more personal-as-political, and relates to an earlier stage of my argument above. In short, if you’re terrified of the potential for AI to reach job-annihilating puissance through the reinforcement of its “learning” through human feedback, why the holy hell are you using it to illustrate your blog posts?
(I’ve found the same paradox in Mark Carrigan‘s content over the last year or so, which oscillates between “we academics are so fucked” and “look how amazing this stuff is”.)
It’s not a well-founded philosophical position, admittedly, nor even a scientific one, but I’m pretty convinced at this point that the greatest application of generative systems is autoevangelism: as a rule of thumb, the more people play with them, the more convinced they are of their puissance and inevitability. The same correlation is pretty obvious in, say, smart-phone usage—and indeed, “AI” has emerged from the same ecosystem that brought us dark patterns and habituation-as-business-model.
Generative models are the fentanyl of capital-C Content. I have seen the greatest minds of my generation &c &c &c.
If the aim is to “stop” this technology—a goal extremely woolly in its formulation, here; how does one propose to put the genie back in the bottle, exactly?—then yes, we will need to motivate “different strands of opposition”. This includes philosophers, but it also includes people for whom most of the points in both the Bear’s post and my response are just abstruse verbiage.
“Autocomplete on steroids”, meanwhile, may be scientifically and philosophically inaccurate, but it is also the sort of everyday concrete metaphor that can do the necessary rhetorical work of activating non-expert opposition. Indeed, it looks like it is already doing so.
I agree that the constant attempts to critique generative models by highlighting the errors in their outputs are tedious and, in the long run, probably self-defeating—but in my case, that’s more because I think that the “it doesn’t actually work as well as claimed” argument has already tacitly agreed to the terms-of-debate offered by the system-builders, and therefore lost the war even as it wins the battle.
The same is true of those who argue “it’s incredible, and it will inevitably get exponentially better”, as the Bear does here.
To write this way is to feed the fire, not to fight it.
Leave a Reply