A New Kind Of Fire

When rubbing two seemingly unrelated concepts together, sometimes you get a new kind of fire.

For example, in the left hand of our contemporary muse:

Il vecchio mondo sta morendo. Quello nuovo tarda a comparire. E in questo chiaroscuro nascono i mostri

The old world is dying. The new one is late in emerging. And in this half-light monsters are born.

—Antonio Gramsci, from his prison notebooks

And in the right hand:

In fact, the overwhelming impression I get from generative AI tools is that they are created by people who do not understand how to think and would prefer not to. That the developers have not walled off ethical thought here tracks with the general thoughtlessness of the entire OpenAI project.

—Elizabeth Lopatto in the Verge article, The Questions ChatGPT Shouldn’t Answer, March 5, 2025

If we rub these particular concepts together, will anything interesting really be ignited? That depends, I think, on whether or not we believe that the old world really is dying, and whether or not we also share the consensus developing among our professional optimists that the technologies we’ve begun to call artificial intelligence represent a genuine hope of resurrecting it.

If we believe that Gramsci’s prescient vision is finally being fulfilled, what monsters, exactly, should we be expecting? The resurgence of fascism and fascists is certainly the one most talked about at the moment, but the new fascism seems to me less like a genuinely possible future, and more like the twitching of a partially reanimated corpse. Vladimir Putin can rain missiles down on the Ukraine all day long, and we learn nothing. Donald Trump can rage and bluster and condemn, and still not conquer France in a month. Despite the deadly earnest of both these tyrants, Marx’s “second time as farce” has never seemed more apt.

The more interesting monsters are the tech bros of Silicon Valley, the idiot savants who modestly offer themselves up to serve as our once and future philosopher kings. They give us ChatGPT, and in their unguarded moments natter endlessly on about AGI and the singularity as though they’d just succeeded in turning lead into gold, or capturing phlogiston in a bell jar.

Never mind asking Sam Altman why, despite his boasting, we haven’t yet achieved AGI on current hardware. Ask a simpler question instead: Why don’t we yet have self-driving cars? More specifically, why don’t we have self-driving cars that can drive anywhere a human being can, in any conditions a human being can? Then ask yourself how you would train a self-driving car to handle all the varied versions of the trolley problem that are likely to occur during the millions of miles driven each day on the world’s roads.

We do know how human beings handle these situations, don’t we? If we train a car’s self-driving system in the same way human beings are trained, should we not expect equivalent results?

The answer to our first question is no, we can’t actually describe with any degree of certainty what mental processes govern human reactions in such situations. What we do know is that mind and body, under the pressure of the split-second timing often required, seem to react as an integrated unit, and rational consciousness seems to play little if any part in the decisions taken in the moment. Ethical judgments, if any are involved, appear purely instinctive in their application.

Memory doesn’t seem to record these events and our responses to them as rational progressions either. When queried at the time, what memory presents seems almost dreamlike—dominated by emotions, flashes of disjointed imagery, and often a peculiarly distorted sense of time. When queried again after a few days or weeks, however, memory will often respond with a more structured recollection. The rational mind apparently fashions a socially respectable narrative out of the mental impressions which were experienced initially as both fragmentary and chaotic. The question is whether this now recovered and reconstructed memory is data suitable for modeling future decision-making algorithms, or is instead a falsification which is more comfortable than accurate.

With such uncertainty about the answer to our first question, the answer to our second becomes considerably more complicated. If we don’t know whether or not the human reactions to an actual trolley problem are data-driven in any real sense, where should we source the data for use in training our self-driving car systems? Could we use synthetic data—a bit of vector analysis mixed with object avoidance, and a kind of abstract moral triage of possible casualties? After all, human reactions per se aren’t always anything we’d want emulated at scale, and even if our data on human factors resists rational interpretation, we already have, or can acquire, a considerable amount of synthetic data about the technical factors involved in particular outcomes.

Given enough Nvidia processors and enough electricity to power them, we could certainly construct a synthetic data approach, but could we ever really be sure that our synthetic data matched what happens in the real world in the way, and to the degree, that we expect? This might be the time to remind ourselves that neither we nor our machines are gods, nor are our machines any more likely than we are to become gods, not, at least, as long as they have to rely on us to teach them what any god worth worshipping would need to know.

I think Elizabeth Lopatto has the right of it here. Generative AI knows, in almost unimaginable historical completeness, what word is likely to come after the word that has just appeared—on the page, over the loudspeaker, in the mind, whatever—but historical is the operative word here. Every progression it assembles is a progression that has already existed somewhere in the history of human discourse. It can recombine these known progressions in new ways—in that sense, it can be a near-perfect mimic. It can talk like any of us, and when we listen to the recording after the fact, not even we ourselves can tell that we didn’t actually say what we hear coming out of the speaker. It can paint us a Hockney that not even David Hockney himself could be entirely sure was a forgery.

The problem—and in my opinion it’s a damning one—is that generative AI, at least as presently designed and implemented, can never present us with anything truly new. The claims made for successful inference algorithms at work in the latest generative AI models seem to me to be premature at best. While it may ultimately prove to be true that machines are capable of becoming independent minds in the same way that human beings do, proving it requires an assessment of all sorts of factors related to the physical and environmental differences in the development of our two species which so far remains beyond the state of the art.

Human beings still appear to me to be something quite different from so-called thinking machines—different in kind, not merely in degree. However we do it, however little we understand even at this late date how we do it, we actually can create new things—new sensations, new concepts, new experiences, new perspectives, new theories of cosmology, whatever. The jury is still out on whether our machines can learn to do the same.

If we’re worried about fascism, or indeed any form of totalitarianism that might conceivably subject our posterity to the kind of dystopian future already familiar to us from our speculative fiction, we’d do well to consider that a society based on machines that quote us back to ourselves in an endlessly generalized loop can hobble us in more subtle but just as damaging and permanent ways as the newspeak and doublethink portrayed in Orwell’s 1984. No matter how spellbinding ChatGPT and the other avatars of generative AI can be, we should never underestimate our own capabilities, nor exchange them out of convenience for the bloodless simulacrum offered us by the latest NASDAQ-certified edgelord.

The bros still seem confident that generative AI can transform itself in a single lifetime by some as yet undefined magic from a parrot into a wise and incorruptible interlocutor. If they’re right, I’ll be happy to consign this meditation to the dustbin of history myself, and to grant them and their creations the respect they’ll undoubtedly both demand. In the meantime, though, I think I’ll stick with another quote, this one by [email protected]‬ in a Mastodon thread about ChatGPT:

I don’t believe this is necessarily intentional, but no machine that learns under capitalism can imagine another world.

The Irrelevance of Precedent

What do I think about TikTok? What do I think about X? What do I think about all our 21st century digital anxieties—China’s nefarious designs on democracy, Musk’s knee-jerk racism, Zuckerberg’s peculiar concept of masculinity, Thiel’s equally peculiar attitude toward his own mortality, and by extension our own?

What I think is that once the box is opened, Pandora can no longer help us—or, in more contemporary terms, scale matters. What does that mean? It means, to resort to the original Latin, Homo sum, humani nihil a me alienum puto. Genuine freedom of speech reveals things to us about ourselves that we’d rather not know. Content moderation can’t help us with that. Neither can the clever pretense of algorithm patrolling, nor bans that, for obvious economic reasons, won’t ever actually be enforced except selectively. Not even some real version of the Butlerian Jihad can help us.

The singularity may never come to pass, but governmental interventions in the creations of the digital age, legislative, executive, or judicial, are, like the military career of Josef Švejk, tainted with all the accidental qualities an indifferent universe can conjure. The truth is, we can no longer afford our own immaturity. My advice is simple: don’t go with the tech bros if you want to live. They really have no idea what they’ve wrought.

Unbidden Bits—December 23, 2024

Historians of the Future:

Frank Herbert’s Dune seems to have been written by a man who’d read too much Gibbon. Max’s DUNE Prophecy, on the other hand, seems to have been created by people who’ve watched too much TikTok.

Viewed from a certain critical perspective, both are satirical masterpieces, and like all such masterpieces, feel eerily appropriate to their times.

Metamorphosis/Metempsychosis

Ten years ago I was privileged to witness the emergence of a dragonfly from its nymph form. A creature that at first glance had seemed like a beetle to my untrained eye had crawled laboriously up from my back yard, attached itself to the side of the concrete step leading to the sliding glass door to my bedroom, and remained there, unmoving, tempting me to believe it had died.

I don’t remember exactly how long it remained there, but it was long enough for me to pass it a number of times on my way back and forth to the alley behind my house. Then, the last time I started to pass it, I discovered to my surprise that the back of its carapace had split open, and the dragonfly it had become was perched atop the empty shell that had sheltered it, had been it, and was now unfolding its wings in the afternoon sun. I watched until the wings, now fully dried, suddenly began to vibrate, then powered a beautiful, iridescent liftoff and arrow-like disappearance into the distance, a movement almost too fast to follow with the naked eye.

Being a child of the 20th/21st centuries, my first thought at witnessing this astonishing sequence of events was that any sufficiently advanced technology will be indistinguishable from magic, my second that there are more things in heaven and earth, Horatio, than are dreamt of in your philosophy. Being a child of the 20th/21st centuries, I didn’t find those two thoughts to be incompatible. Now, ten years later, I still don’t.

Die Würde des Menschen ist unantastbar*

*Human dignity is inviolable (The first sentence of Article I of the postwar Grundgesetz für die Bundesrepublik Deutschland (Basic Law for the Federal Republic of Germany)

If we actually wanted any further proof of Santayana’s contention that “those who cannot remember the past are condemned to repeat it,” we’d need look no further than the current resurgence of fascist sentiment in Europe and the United States, and the rising support worldwide for authoritarian governments devoted primarily to exclusion, punishment, and degradation.

The Germans, of all people, should know better. They did know better in 1949, when their remarkable postwar constitution was written and enacted in the western half of their still divided country. If the strutting members of the AfD have already forgotten what motivated their great-grandparents to enshrine Article I as the only article which by law cannot ever be amended or repealed, there seems little hope that any of the rest of us will remember our considerably more ambiguous commitments to the same principle.

There are reasons why the Nazis are back in Germany, why a paranoid and vicious authoritarianism is once again the shiniest of political baubles everywhere in the world. As comforting as it might be for those who dread what’s coming to think so, none of these reasons can be attributed solely to the historical blindness of the generations born since the end of World War II. The truth of the matter is that neither democracy nor capitalism, as practiced by our supposedly enlightened postwar governments, has ever been overly concerned with the dignity of all human beings.

It should surprise no one who’s been paying attention that in the nearly eighty years since the end of World War II, the distribution of wealth and power in western democracies has gradually come to resemble that of some of the worst hierarchical societies of the past. The disenfranchised, dispossessed, and disenchanted armies of the underemployed and unrepresented are back with us, and they will absolutely not be mollified any more easily by our well-meant homilies about human dignity than they were in 1933. As right-wingers in the United States like to say, the die is cast. It’s hard to see how there’s any good news in that for anybody.

Angels in America

Once upon a time in California, I was late getting getting home from work on election day, and had just enough time to grab my sample ballot and leg it to my local polling place two blocks away before it closed. As I hustled past a lifted Ram pickup with a chrome bull bar idling menacingly in the mouth of my local gas station driveway, the driver, a young man in a ten gallon Stetson and sunglasses, flashed the lights at me, stuck his arm out of the driver’s side window and slapped the outside of the door.

“¡Andale Viejo!” he belted out. “¡Que te vaya bien!“ I gave him a perfunctory thumbs up and kept on trucking.

¿Viejo? I grumbled to myself. I’m forty-one, for fuck’s sake!

He was right, though. I’ve been old since I was ten, but now I’m eighty, and still hobbling along just fine. Go figure. Maybe that cowboy benediction had something to do with it. I’d certainly like to think so….

A Brief Reminder

Our grandchildren aren’t stupid. Their mental equipment isn’t inferior to ours. They just live in a different world, one which no longer belongs to us even though we helped create it. It’s theirs now, and whatever we imagine, we’re no longer in any position to judge them. Likely they’ll be fine, but if they turn out not to be fine, it’s going to be very hard to show how listening to us would have made the slightest bit of difference.

The Enlightenment’s Farewell Tour

In the U.S., the Republicans’ sad entourage of the desperate, demented, and enraged are tearing at the Constitution’s exposed achilles tendons. In Russia the gangsters of Prigozhin are battling the siloviki of Putin for control of the spoils of a twice-failed totalitarian state. In India, Hindutva pursues a scorched-earth battle against Islam. In Germany the AfD tidal wave has engulfed the SPD, broken the CDU, and arrived at last in Bavaria, intent on washing away once and for all what little is still left of the CSU’s liberal democratic pretensions.

In Italy a fascist consumerism has sprung full-grown from the brow of Meloni. The trains now run on time, and foreign investors are once again reassured. In Finland and Sweden, the local populists have decided that white people are the only real people after all. In Israel, Syria, Hungary, Belarus, and Turkey, the warlord grifters have outlasted everyone. In Saudi Arabia and the gulf states, the kings, emirs, sultans and satraps of one kind or another are now completely convinced that having more money than Allah the Merciful means not having to apologize to anyone ever.

In Iran, a cabal of wizened religious fanatics calling themselves the Islamic Republic have yet to see any reason to deny themselves the perverse pleasure of beating and imprisoning women at random, and of shooting their own children whenever the kids act like they might be the coming thing. In China, a suspiciously but undeniably prosperous Communist (sic) Party oligarchy has decided that the Universal Declaration of Human Rights is nothing more than a confession of the failures and arrogance of so-called Western civilization.

I don’t think Web 3.0 is going to be a lot of help in preserving what’s left of the secular humanism that evolved over four centuries in Europe, and was sealed with the French and American revolutions. Neither will eleven aircraft carrier battle groups or a triad of delivery systems for nuclear weapons, given that both have long been controlled by people who bear no allegiance whatsoever to secular humanism either as a creed or a philosophy of government.

I’m not one to cry O tempora, o mores! every time someone in Washington does something stupid, but I do think that if the last sad road show of the Enlightenment comes to your town, you should go and listen to what was once promised us, and what very shortly we’ll all be missing.

The Center for AI Safety’s Statement of AI Risk

The concern expressed in The Center for AI Safety’s Statement of AI Risk seems justified to me, but it also seems to me that many of the signatories have still not grasped the real nature of that risk. It’s the second order effects that’ll do us in—not the singularity and its presumptively implacable AI overlords, but rather the symbiotic processes already inherent in pervasive computing, processes which we can all sense, but are still in denial about what it will take, in terms of an evolution in human consciousness, to successfully navigate those spaces which still exist between where the machine ends and we begin.

In his 1960 Critique de la Raison Dialectique, Jean-Paul Sartre indulged himself in a typically poetic digression about how we can’t tell—may never be able to tell—whether we’re dreaming the machine, or the machine is dreaming us. This is a commonplace now, but although it wasn’t entirely new in 1960, it was still controversial enough to meet with widespread ridicule among the opinion makers of the day. And of course Sartre was describing the strictly physical interactions of humans and industrial age assembly lines, when machines were dumb, and humans were still thought to be the masters no matter how deeply their own mental processes were conditioned by the mechanical repetitions of their jobs.

The machines today are no longer dumb, and we can no longer afford the illusion that we are the masters of either the physical or the mental aspects of the machine/human symbioses of the 21st century. I’m not sure why, but I’m not as bothered by this as the signatories of this letter are telling me I ought to be. It certainly isn’t because I’m an optimist in the narrow sense ot the term. I expect great darknesses in our future, but not the ones that are supposedly keeping the tech bros up at night. These latter day idiot savants aren’t the real heralds of our new distempered age, it’s the kids now glued to TikTok all day. What their stewardship of our future will look like remains beyond anyone’s current power to predict. To make a long story short, it’s not the end of humans that should concern us, but the end of humanism, which seems to be losing its grip on the tiller of this ship of fools we’re crewing well before a new helmsman is ready to take its place.

Agency

The first of many beginnings that turned out to have no middle or end. Waste not, want not, though, right?

It was social services placed me here, in this two-person, three-yappy-dog suburban coffin, here to prosper and grow up, after which they’re presumably going to let me out into the world again. As if I can afford to wait that long. They’re good people, nice people, these two, but they’re not my people. Do I even have people? Doesn’t feel like it, not so far.

So I’m on my own now, is it? Better not lose my library card then. I’ll be needing it for planning and stuff.