My flirtation with so-called smart home technology began with internet connected cameras that allowed me to see inside my house when I was away on trips, and with locks that I could unlock remotely in case my neighbors needed access in emergencies. Maybe the memory of my mom driving my dad crazy by wondering an hour or so into every Sunday drive if she’d left the iron on is what set me up for this. Whatever it was, I found that the offer of a personal panopticon to help control the things in my life that might still be controllable was genuinely seductive, even though I’d long been aware that the things one can control in life aren’t the things that actually matter.
In any event, ten years or so into this adventure, the technology had matured to the point that relying on it to help me do things I hadn’t previously been able to do seemed quite natural. Then the daemon of generative AI sprang full-grown from Sam Altman’s brow—unbidden, unannounced, largely unfathomable—and demanded to be a full partner in the pilgrim’s progress I had up till then considered to be my private life.
Would you like me to show you what movies are on that you really want to watch? Here’s all the news I compiled for you today—I left out the things I know would upset you. I saw that you were out of coffee, so I ordered ten pounds of your favorite whole beans from that new place you got the push notification from last Tuesday, the one you saved in your shopping list.
No, thank you. Raped by the heralds of demented late-stage capitalism was definitely not something I wanted chiseled on my tombstone, let alone tattooed on my backside. Was there any way to avoid it, though? Well, not entirely, but there was Apple.
Apple was (and still is) almost universally considered among the technorati to be hopelessly behind in AI, particularly in the generative AI that made Alphabet’s Gemini so much smarter and more responsive than Siri, so much more competent at the agentic functions that made a personal assistant genuinely useful. Fine, I thought, investigating further, I’ll stick with Apple, then. They promise they’ll let me turn the good shepherd stuff off, and retain at least the illusion of free will.
I should have known better. I don’t think anyone has yet realized just how demeaning, yet inescapable, our dance with the agents of virtual personhood is going to be. For example:
I like to read before going to sleep, but I don’t like having to get out of bed to go turn the lights off when I’m finally ready to put away the reading and get some shuteye. Most people probably just switch off their bedside lamps, but given the built-in recessed ceiling lights that came with my house, I skip the bedside lamps and instead engage in a three-part Siri conversation with a HomePod on the other side of the room:
Siri, good night. This turns off all the lights except the lights above the bed, locks all the doors, turns off the TV and speakers in the living room if they’re still on, and checks to see that the garage door is closed.
Siri, before bed. This sets the lights above the bed to the right color temperature and brightness for reading.
Siri, bedtime. This turns the lights above the bed off.
A couple of nights ago, I mistakenly began the sequence by issuing the second request, before bed, instead of the first, good night. Siri, however, responded as though I’d actually said good night. Since the two requests don’t actually sound anything alike, I’m tempted to believe that the Siri algorithm(s) have taken note not only the content of the requests I’ve been issuing almost every night for the last few years, but also their sequence, and very helpfully did what it assumed I wanted it to do instead of what I actually asked it to do. The fact that its inference was helpful in this particular situation didn’t keep it from feeling like a scenario straight out of 1984. These are precisely the sorts of judgments that no one who values their personal autonomy wants a stochastic parrot to be making, even in support of the seemingly benign act of turning lights off and on.
Get ready folks. Unlike me, you might not have to share your bed with them, but it does look like these corporate nursemaids are going to be looking over everybody’s shoulder from now until some future Sam or Elon decides there’s more profit in thermonuclear war, desertification, or Soylent Green. Last one to the singularity is a rotten egg!
