Nuclear war, AGI, and the importance of understanding what makes an existential risk.
Four years ago, I wrote one of my most controversial articles. It argued that climate change — while it will make the world we live in much worse and lead directly and indirectly to the deaths of millions — won’t end human life on Earth.
This isn’t scientifically controversial. It’s consistent with IPCC projections, and with the perspective of most climate scientists. Some researchers study extreme tail-risk scenarios where planetary warming is far more catastrophic than projected. I think studying that is worthwhile, but these are very unlikely scenarios — not anyone’s best guess of what will happen.
So the reason that arguing that climate change is likely not a species-ending threat is so controversial isn’t because of science. It’s because the argument can feel like intellectual hair-splitting and hand-waving, like a way of diminishing the severity of the challenge that unquestionably lies ahead of us.
Millions of people will die with climate change, and that’s horrendous; it feels almost like selling those victims out to tell comfortable people in rich countries that they will probably not be personally affected and will probably get to continue in their comfortable lives.
But fundamentally, I believe in our ability to solve problems without exaggerating about them, and I don’t believe in our ability to solve problems while exaggerating about them. You need a clear picture of what’s going to happen to fix it. Climate action addressed with the wrong understanding of the threat is unlikely to save the people who actually need saving.
AI, nuclear war, and the end of the world
This has been on my mind recently as the case that AI poses an existential risk to humanity — which I’ve written about since 2018 — has gone mainstream.
In an article in Time, AI safety researcher Eliezer Yudkowsky wrote that “the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die.” New international treaties against building powerful AI systems are part of what it’ll take to save us, he argued, even if enforcing those treaties means acts of war against noncomplying nations.
This struck a lot of people as fairly outrageous. Even if you’re convinced that AI might be quite dangerous, you might need more convincing that it’s extraordinarily deadly indeed to consider it worth risking a war. (Wars are also dangerous to the future of human civilization, especially wars with the potential to escalate to a nuclear exchange.)
Yudkowsky doubled down: Uncontrolled superhuman AI will likely end all life on Earth, he argued, and a nuclear war, while it would be extremely bad, wouldn’t do that. We should not court a nuclear war, but it’d be a mistake to let fear of war stop us from putting teeth in international treaties about AI.
Both parts of that are, of course, controversial. A nuclear war would be devastating and kill millions of people directly. It could be even more catastrophic if firestorms from nuclear explosions lowered global temperatures over a long period of time, a possibility that is contested among experts in the relevant atmospheric sciences.
Avoiding a nuclear war seems like it should be one of humanity’s highest priorities regardless, but the debate over whether “nuclear winter” would result from a nuclear exchange isn’t meaningless hairsplitting. One way we can reduce the odds of billions of people dying of mass starvation is to decrease nuclear arsenals, which for both the US and Russia are much smaller than they were at the height of the Cold War but are recently on the rise again.
Is AI an existential risk?
As for whether AI would kill us all, the truth is that reporting on this question is honestly extraordinarily difficult. Climate scientists broadly agree that climate change won’t kill us all, though there’s substantial uncertainty about which tail-risk scenarios are plausible and how plausible. Nuclear war researchers have substantial, heated disagreement about whether a nuclear winter would ensue from a nuclear war.
But both of those disagreements pale in comparison to the degree of disagreement over the impacts of AI. CBS recently asked Geoffrey Hinton, called the godfather of AI, about claims that AI could wipe out humanity. “It’s not inconceivable, that’s all I’ll say,” Hinton said. I’ve heard the same thing from many other experts: Stakes that high seem to be genuinely on the table. Of course, other experts insist there is no cause for worry whatsoever.
The million-dollar question, then, is how AI could wipe us out, if even a nuclear war or a massive pandemic or substantial global temperature change wouldn’t do it. But even if humanity is pretty tough, there are many other species on Earth that can tell you — or could have told you before they went extinct — that an intelligent civilization that doesn’t care about you can absolutely grind up your habitat for its highways (or the AI equivalent, maybe grinding up the whole biosphere to use for AI civilization projects).
It seems extraordinarily difficult to navigate high-stakes trade-offs like these in a principled way. Policymakers don’t know which experts to turn to to understand the stakes of AI development, and there’s no scientific consensus to guide them. One of my biggest takeaways here is that we need to know more. It’s impossible to make good decisions without a clearer grasp of what we’re building, why we’re building it, what might go wrong, and how wrong it could possibly go.
A version of this story was initially published in the Future Perfect newsletter. Sign up here to subscribe!
The viral fake Drake and The Weeknd song tells us a lot about the future of AI.
You’ve probably heard a lot lately about AI.
Everyone from Elon Musk to Joe Biden has been worried that AI could take over our jobs, spread misinformation, or even — if we’re not careful — one day kill us all. Meanwhile, some AI experts say instead of fixating on hypothetical doomsday scenarios in the long term, we should focus on how AI is actively harming us right now and the concentration of power in a handful of companies that are controlling its development. Already, the error-prone technology has been used to invent slanderous lies about people, hack bank accounts, and mistakenly arrest criminal suspects.
But near-term and long-term concerns aside, there’s a major component to why it feels like AI is suddenly taking the world by storm: It’s fun.
Over the past few weeks, I’ve been playing around with the latest AI tools and talking to people who use them. I’ve found that the most exciting forms of AI right now are not the kind people are using to increase productivity by crunching spreadsheets or writing emails. (Although bosses love that idea!) They’re the kind being used to entertain us.
In just the past six months, AI has come an incredibly long way in helping people essentially create all kinds of media. With varying degrees of instruction, AI can craft photorealistic illustrations, design video games, or come up with catchy tunes with top-40 potential.
So what should we make of the fact that people are enthusiastic about using a technology that clearly has serious flaws and consequences?
“I think it’s completely reasonable for people to be excited and having fun,” Margaret Mitchell, chief ethics scientist for AI platform Hugging Face, wrote in a text. Mitchell formerly founded the Ethical AI team at Google, where she was controversially fired after co-authoring a paper calling out the risks associated with large language models that power many AI apps. Mitchell and her co-authors were prescient early critics of the shortcomings of recent AI technology — but she acknowledges its potential, too.
Pietro Schirano is a design lead at financial services startup Brex. He was also an early adopter of GPT-4, the latest iteration of the technology from the company behind the viral ChatGPT app, OpenAI.
When GPT-4 came out in March, Schirano couldn’t wait to use it. He decided to test its ability to write working lines of code from simple prompts. So Schirano set out to recreate the video game Pong because, in his words, “it was the first video game ever, and it would be cool to do it.”
In less than 60 seconds, after feeding GPT-4 a few sentences, copying the code, and pasting it into a code engine, Schirano had a working Pong he could play with. He was amazed.
“That was the first time that I had this sort of, like, ‘oh shoot’ moment where I’m like, oh my god,” he said. “This is different.”
His tweet posting a video of the process went viral.
When I asked Schirano if he worries about AI replacing the jobs of people like him, he said he wasn’t too concerned. He says he uses ChatGPT at work to help him be more productive and focus on higher-level decision-making.
“The way that I see these tools is actually not necessarily replacing us, but basically making us superhuman, in a way,” said Schirano.
As my colleague Rani Molla has reported, many workers are in the same camp as Schirano. They don’t think their jobs can fully be replaced by AI, and they aren’t particularly terrified of it — for now.
I talked to Ethan Mollick, a professor at the University of Pennsylvania’s Wharton business school, about the wide range of reactions to new AI tools. Any kind of new “general purpose technology” Mollick said — think electricity, steam power, or the computer — has the potential for major disruption, but it also catches on because of unexpected, novel, and often entertaining use cases. New advancements in AI like GPT-4, he added, fit very well into that general-purpose technology category.
AI is “absolutely supercharging creativity,” said Mollick. “How can you not spend every minute trying to make this thing do stuff? It’s incredible. I think it could both be incredible and terrifying.”
These fun creative ways to use AI also bring up the question of authenticity: Will it replace human creativity or merely help us in the production of it?
Last week, a hip-hop song that sounded like a mashup of the artists Drake and The Weeknd went viral on social media. The song, posted by the anonymous user “ghostwriter,” claimed to be made with AI, and got millions of plays before it was taken down by major platforms.
The proliferation of this kind of AI-generated media has spooked record labels enough that Universal Music Group asked streaming services like Spotify to stop AI companies from using its music to train their models, citing intellectual property concerns. And last month, a coalition of record industry unions and trade groups launched the “Human Artistry Campaign” to make sure AI doesn’t “replace or erode” artists.
A few artists, though, have embraced the AI concept. The musician Grimes even asked fans to co-create music with her likeness and offered to split 50 percent of the royalties.
Mollick compared the debate around whether AI will replace artists to the introduction of the synthesizer to modern music. When the synthesizer first came out, people debated whether it was “ruining music,” and whether people who used the instrument were real musicians.
Ultimately, the real dangers of AI may not lie so much in the technology but in who controls it and how it’s used, Mitchell said.
“My issues are more with tech leaders who mislead and push out tech inappropriately rather than creative people exploring new technology,”
A version of this story was first published in the Vox technology newsletter. Sign up here so you don’t miss the next one!
A new movie adaptation captures the sneaky complexity of what Judy Blume’s classic gets right about being 11.
In my mental catalog of Judy Blume books, everything is filed according to the adolescent trope it taught me about. It’s Not the End of the World is the divorce book. Forever… is the sex book. Then Again, Maybe I Won’t is the wet dreams book, which in the fifth grade read like an intelligence report sent from behind enemy lines. Are You There God? It’s Me, Margaret, first published in 1970, is, of course, the period book.
Or at least that’s how I remembered it from being 10 years old: Margaret and her friends competing to see who will get their periods first; that awful Nancy lying about getting it and then crying in the bathroom of a fancy restaurant when it actually came; Margaret alone in her room, fumbling with the mysterious belt that sanitary napkins used to have in the ’70s. Girls read Are You There God as a supplement to middle school health classes, was what I vaguely concluded, so they would know what it felt like when their periods came. (I could not imagine a boy reading this book for any reason whatsoever.)
So I was startled, when I watched Kelly Fremon Craig’s excellent new film adaptation of Are You There God, to see that there were so many other plotlines at work in the story. There was all this stuff about bras, which I did vaguely recall. But then there were also friendship dynamics and family dynamics and new town stuff and religion stuff — my god! How had I forgotten the religion stuff? It’s right there in the title!
It didn’t make sense to me. The damn book is only about 30,000 words long. Could Blume possibly have fit that much in there besides the period plotline? Had Craig added something new?
Not really, I discovered upon returning to the book for the first time in 25 years. Nearly everything in Craig’s film is in Blume’s book, too. As it turns out, Are You There God? It’s Me, Margaret is a lot more than just the period book. It hits all the tweenage low points, and it welcomes adult readers with a surprising amount of grace.
Are You There God begins with Margaret Simon, 11 years old, moving out of New York’s Upper West Side and into the suburbs of New Jersey. In a gently understated subversion of the city kid stereotype, she finds when she arrives that she seems younger for her age than the jaded suburbanites all around her. Queen Bee Nancy, who immediately takes Margaret under her wing, already plays with makeup and practices kissing boys, while Margaret does neither. Nancy seems, Margaret notes, impressed and resentful, “to know a lot.”
Toxic girl friendships are one of the great themes of Blume’s oeuvre (explored in their darkest form in the harrowing Blubber). Accordingly, once Nancy enters the picture, Are You There God becomes the story of Margaret’s education and corruption. Nancy pushes Margaret to buy a training bra and sanitary napkins, teaches her to perform her crushes, tells her which boys are acceptable crush objects. She plies Margaret with cruel gossip to turn her against the girl in their class who already wears bras.
It’s Nancy who informs Margaret that her secular existence is a problem. The Simons identify as neither Jewish nor Christian after Margaret’s mother, Barbara, was disowned by her Christian parents for marrying a Jewish man. In New York this wasn’t an issue, but in the suburbs, Nancy tells Margaret, everyone joins either the Y or the Jewish Community Center.
Desperate to follow Nancy’s instructions, Margaret embarks on a research project to try to find what religion is best for her. She isn’t looking to be shown how to talk to God: She already does that every night, alone in her room, pouring out her dreams and worries to God as though she’s writing in her diary. (“Please help me grow God. You know where.”) Instead, Margaret wants organized religion to codify her relationship with God, to turn it into something that Nancy will be able to categorize neatly away in the same way she does everything else.
Picking a religion wouldn’t only help Margaret with her school friends. It would also help her with her family. Simmering mostly below the surface of Margaret’s naive narration is a proxy war between her parents and her grandparents, with Margaret’s life as the battlefield.
Margaret is devoted to her paternal grandmother, Sylvia, who dotes on her, paying for her New York City private school tuition and her New Hampshire summer camp. Sylvia feels fiercely that Margaret should grow up as a New York Jew, in culture if not in religious belief, and to that end shows up uninvited at the Simons’ new house with bags of deli food to make sure Margaret will remember what it should taste like. “Mmm … nothing like the real thing!” she says every time she bites into a pickle.
Margaret is vaguely aware that her mother doesn’t particularly like for Sylvia to be so involved in Margaret’s life, and she’s pretty sure that they moved to New Jersey specifically to get Margaret away from Sylvia. She’s less aware of how many times Sylvia tells her to do something and then follows it up by instructing her not to tell her parents. It all explodes, though, when Barbara’s parents show up in town to check up on their granddaughter, forcing Margaret to miss a planned visit to Sylvia.
“Margaret is a Christian!” says Barbara’s mother.
“You’re a Jewish girl,” Sylvia assures Margaret.
“I’m nothing, and you know it!” proclaims Margaret. “I don’t even believe in God!” (To us she adds, “I wanted to ask him did he hear that!” She is, at the time, in a fight with God.)
It’s this plotline that Craig gently expands, mostly through transforming Barbara (played here by Rachel McAdams at her warmest) into a point-of-view character. Craig’s take, though, is less an addition to Blume’s text than an excavation, working with what Margaret tells us to unearth new details in the story.
Blume’s Barbara is an artist who seems a little scattered in the suburbs and lightly on edge around her mother-in-law. Craig’s Barbara, in turn, is a bohemian who moves to the suburbs because she thinks it will be best for her child and then finds herself losing her own identity; who would like her mother-in-law to stop trying to raise her daughter for her now, please. In the film, Barbara becomes a stand-in of sorts for all the women who grew up on Judy Blume and are watching Craig’s film now as adults. Through Barbara’s eyes, we watch Margaret care passionately about things like not wearing socks with loafers and buying a bra well before she needs one, and we remember what it was like to care so passionately.
All of it is consistent with the characterization Blume gave us, back when she wrote Margaret as a newly suburban housewife herself, writing books in between caring for her children. It’s just the stuff that Margaret, with the complacent solipsism of childhood, never bothers to lay out for us. She’s too busy trying to figure out how to be the version of herself Nancy and her parents and her grandparents want her to be — until she figures out how to be a version of herself that she likes instead.
Once she figures it out, you know what happens next: She gets her period.