Fake AI Podcasters Are Reviewing My Book And It Scares Me

As someone who has been following the growth of generative AI for a while now, I know that the technology can be pretty good (if not quite human-level) at quickly summarizing complex documents into a more digestible format. But I wasn't yet prepared for how captivatingly compelling it would be to hear Google's NotebookLM condense my recent book on Minesweeper in a short, 12.5-minute, podcast-style conversation between two people who don't exist.

There are still enough notable issues with NotebookLM’s audio output to prevent it from completely replacing professional podcasters in the near future. Still, the podcast-like format is an incredibly engaging and endearing way to digest complex information and points to a much more pleasant future for generative AI than the boring back-and-forth of a text-based chatbot.

Hey! Listen!

Listen to NotebookLM's 12.5 minute summary of my Minesweeper Book using the player above.

Over a year ago, Google launched NotebookLM as “a virtual research assistant that can summarize facts, explain complex ideas, and spark new connections — all based on the sources you select.” However, last week, Google added the new “Audio Summarization” feature, which it sells as “a new way to turn your documents into engaging audio discussions.”

Google doesn’t use the word “podcast” anywhere in that announcement, instead talking about audio creations that “summarize your material, draw connections between topics, and trade pleasantries.” But Wharton AI professor Ethan Mollick correctly referred to the style as a “podcast” in a recent social media post sharing a NotebookLM audio overview of his book. Mollick called these audio summaries “the best current demonstration of AI that makes you think ‘wow, this is awesome and useful!'” and “baffling, too” — and we agree on both counts.

Inspired by Mollick's post, I decided to submit my own book to NotebookLM to see what its virtual “podcasters” would make of 30,000 words about the classic 1990s Windows gaming game. Minesweeper (Believe it or not, I could have written a lot more.) Just a few minutes later, I was experiencing a reasonable replication of what it would be like to appear on NPR's Pop Culture Happy Hour or a similar joke-filled podcast.

Just the facts?

The NotebookLM summary touches on all the main sections of the book: the prehistory of the games that inspired Minesweeper; the uphill battle over the Windows Entertainment Pack at a business-focused Microsoft in the 1990s; the moral panic over the game's pre-installation on millions of corporate and government computers; and the surprising cheating controversies that surrounded the game's competitive scene.

Sure, I could argue about which specific parts I decided to highlight and/or leave out of the summary (perhaps including the various chapters individually would have led to more detailed summaries), but anyone who listens to this “podcast” will get the same overview of my book as if they were to listen to one of the many actual podcasts I wrote after the book's release.

While there weren't complete hallucinations in the NotebookLM summary “podcast,” there were a few points where they got small details wrong or made assumptions that weren't supported by the text. Minesweeper predecessor Mine exhaustedFor example, NotebookLM's audio summary says, “So this is where those squares and flags start to come into play…” although Exhausted mine It had no features.

Then there's the part where the summary mentions a senator who called Minesweeper “A menace to the republic,” repeating the quote for emphasis. That definitely captures the spirit of Senator Lauch Faircloth's tirade against Minesweeper and other games that are pre-installed on government computers. However, in the context of the podcast, it seems the voices are putting words in Faircloth's mouth by sharing a direct quote.

Small, glaring errors like these (and a few key bits of the book not included in the podcast) would give me pause if I were trying to use a NotebookLM summary as the basis for an academic paper or journalistic piece. But I could use a summary like this to get a quick Cliff's Notes-style primer on a thick tome I didn't have the time or inclination to read in full. And, unlike poring over Cliff's Notes, the concise, podcast-style format would actually make for pleasant background noise while I'm out for a walk or running errands.