Brave New Bookshelf Episode 73: Revolutionizing Multi-Voice AI Narration with Phil Marshall and Stacy Smith Rogers from Spoken

Audiobooks can open up new ways for authors to reach readers, but producing them has traditionally required significant time, money, and expertise. AI-powered narration is beginning to change that, particularly as tools move beyond simple single-voice narration toward more complex, multi-voice productions.

In Episode 73 of the Brave New Bookshelf podcast, hosts Steph Pajonas and PublishDrive Community Manager Danica Favorite spoke with Phil Marshall, CEO and founder of Spoken, and Stacy Smith Rogers, who leads Author Relations at Spoken, about the company's approach to AI audiobook production.

Their conversation explored multi-voice narration, voice cloning, audiobook production workflows, and how authors can maintain creative control while using AI to bring their books into audio.

Apple Podcasts | Spotify | Amazon Music | YouTube | iHeartRadio | RSS Feed

Making Multi-Voice Audiobooks More Accessible

One of Spoken's main areas of focus is multi-voice narration.

Instead of using one narrator for an entire book, authors can create productions in which different voices are assigned to different characters. This can include duet narration, such as using different voices for male and female characters, or larger multicast productions where individual characters have their own voices.

Phil explained that Spoken was built around the idea that both single-narrator and multi-voice production could become more accessible to authors.

The challenge, however, is not simply generating different voices. Those voices still need to work together as part of one coherent listening experience.

Spoken describes this as narrative cohesion, ensuring that a multi-voice audiobook feels like a unified performance rather than a collection of disconnected audio clips.

The company is working on its MV2, or Multi-Voice Version 2, release, which Phil said is designed to improve this process further.

Casting AI Voices for Characters

Spoken's workflow can analyze a manuscript and identify individual characters.

Authors can then choose how those characters should sound. The platform can generate custom voices based on characteristics found in the manuscript, including tone, inflection, and accent.

Alternatively, those characteristics can be used to find suitable voices from Spoken's voice actor library.

Phil gave an example from his own science-fiction novel, which contains more than 100 speaking characters. Using Spoken, he was able to assign voices designed around the individual characters he had written.

This approach can be particularly useful for books with large casts, as well as genres where listeners may want a more immersive audio experience.

Using Your Own Voice Without Narrating the Entire Book

AI voice cloning also creates another option for authors, particularly those writing nonfiction and memoir.

Stacy discussed the importance of hearing an author's own voice in these types of books. With voice cloning, authors can create a digital version of their voice without having to manually narrate an entire manuscript.

Authors can also combine digital narration with real recordings.

For example, Stacy described recording a personal introduction in her own voice before the digitally narrated portion begins. This gives authors another way to maintain a personal connection with listeners while still using AI for the main production.

Authors can keep a cloned voice private for their own projects or, through the options discussed in the episode, make a voice available for other creators to use.

A "Pay When Perfect" Approach

One concern with generative AI tools is paying for outputs that ultimately need to be regenerated.

Spoken has taken a different approach with what it describes as "free to use, pay when perfect."

Authors can work on their audiobook, listen to the results, adjust their choices, and regenerate passages before paying for the finished production.

The process begins by uploading a manuscript. Spoken supports EPUB and Word files, as well as PDFs for use cases such as screenplays. Shorter works can also be entered directly.

The platform then analyzes the manuscript through a process Spoken calls Fabulate. This can generate chapter summaries, identify comparable works and tags, analyze the type of story, and suggest appropriate narration approaches.

For multi-voice projects, the system can identify characters and help generate or select voices for them.

Authors can then use a lexicon to address words or phrases with uncertain pronunciations by providing phonetic alternatives.

Once the voices and pronunciations are ready, the audiobook can be generated and proofed. Authors can listen to the result, make adjustments, upload cover artwork, and add introductory material before publishing.

Payment only takes place when the author is ready to publish.

At the time of the podcast, Spoken's stated price was $10 per 5,000 finished words for subscribers and $20 per 5,000 finished words for non-subscribers.

AI Audiobooks Still Need Human Input

A recurring theme throughout the conversation was that AI audiobook production is not simply about pressing a button and accepting whatever comes out.

Authors still need to listen.

They need to decide whether voices fit their characters, whether pronunciations are correct, whether pacing works, and whether the performance reflects what they intended when they wrote the book.

Spoken has introduced its Certified Spoken Producer Program for authors who either do not have the time to manage that process themselves or do not feel confident making audio production decisions.

The program trains people to work inside the Spoken studio and help authors produce their audiobooks. Stacy explained that this can create opportunities for virtual assistants, author assistants, audio professionals, and voice actors while giving authors another option for managing production.

For authors with larger backlists, working with a producer could also make it easier to bring previously published titles into audio.

Keeping Voices Consistent Across a Series

For authors writing a series, character consistency can become particularly important.

Spoken allows custom-generated voices to be saved for future use. This means an author can return for the next book in a series and use the same voices for recurring characters rather than beginning the casting process again.

That can reduce some of the work involved in producing subsequent audiobooks while helping maintain continuity for listeners.

Putting the Human Story Behind AI Narration

Phil and Stacy also discussed Your Story, a contest created by Spoken in collaboration with Author Nation.

Authors submitted works of under 10,000 words that had been digitally narrated through Spoken. But the contest did not focus only on the finished audio.

Participants were also asked to explain why they were uniquely positioned to tell their particular story.

The aim was to highlight the people behind AI-assisted creative work and reinforce the idea that the technology is being used to deliver a human-created story.

Finalists were evaluated based on the story itself, the audio performance, and the author's personal story behind the work. The grand prize included a cinematic book trailer created by Spoken's in-house video team.

Where AI Audiobook Production Is Heading

The discussion with Spoken highlights how quickly digital narration is moving beyond basic text-to-speech.

Authors can now experiment with different narrators, cast individual characters, clone their own voices, control pronunciation, create personal introductions, and build multi-voice productions without following the traditional full-cast production model.

At the same time, the episode makes clear that the author's role remains important.

Technology can generate the audio, but authors and producers still make the creative decisions that determine how a story should sound.

For independent authors looking at their next format or considering how to give an existing backlist another life, developments in AI narration are creating new options for turning written stories into audio.