11
Oct

How Can Writers And Publishers Utilise Artificial Intelligence (AI)

Putting aside the constant scare of whether robots will soon take our jobs – coming even from creatives -, AI is already here and more and more companies are using it. But what does AI stand for and how can writers and publishers utilise artificial intelligence? This article starts with a little history for newcomers, then explores some solutions that are already existent and available for all and poses some questions about the future.

This year’s Frankfurt Book Fair is all about innovation and technology. On this note, let’s explore what AI can do for publishers, and whether there is anything publishers can do for AI.

Artificial Intelligence: Definition and brief history

Audience analysis and engagement

Content analysis, recommendation and creation


Artificial Intelligence: Definition and brief history

We don’t tend to be very careful when it comes to calling something intelligent: from laundry liquids to nutrients, we use this adjective without giving it a second thought. When it is about computers becoming intelligent, however, we suddenly become much more cautious and self-aware. There is nothing to be worried about, though. A computer can’t become intelligent in the sense a human can: AI is only a machine that is programmed to take its surroundings into account when solving a problem and ‘learn’ as it goes. While the scope of what capabilities are considered AI vary as the technology evolves, it currently involves natural language processing and production, understanding speech, see things (visual recognition) and solving strategic problems (military or games).

In the era when autonomous cars have become not only a possibility but a reality, is hard to imagine that AI research has started in the 1950s – although the idea was present much earlier in science-fiction, as most of our readers probably know. Not surprisingly, the first substantial funding came from the Department of Defence and the development for a while was focused on using AI in military, medical and finance contexts – and solving chess problems.

When I woke up this morning, however, it was AI that filled my feed with articles I will want to read, it was AI that filled my inbox with well written and creative sales emails I didn’t want to read but did anyway, and it was AI that recommended me to take the bus instead of the tube today on my daily commute.

So, how does it work?

In theory, it is incredibly simple: you feed in lots of data, the machine recognizes a pattern and creates an output. For example, Google Maps knows everything that is happening in London today, traffic jams, planned road closures, cycling events and pipe bursts. They know these things partly by scanning the whole world with cute little cars, partly by all the location information Android users consciously and not so consciously share with them. It also knows where I live and work. I shared this information with Google willingly – curious, how data security and privacy policy (GDPR) will change with the new EU legislation being effective from May 2018. By analysing these patterns and the available options in a fraction of seconds, it can list the most favorable options of getting to the office today: if I had to collect all this information myself, I would probably still be at home. AI learns from feedback and readjusts its algorithms to a better little soldier: the more you use a particular system, the better it gets.

The broader category, artificial intelligence also includes machine learning and deep learning. Machine learning means the capability of an AI system to change based on feedback or new data input and generate new algorithms on its own. Deep learning, on the other hand, is trying to ‘mimic’ the way human brain works: there is an input and an output and the processing is done nonlinearly, on many layers.

This also means that there is no AI that can do everything: there are separate little systems specialized to solve one problem or other. As a publisher, you’ll have to decide which problems you can solve with current AI solutions and which ones to implement.

Use it or lose it

Apart from making our lives easier in general, how can AI help publishers?

Simplifying the publishing process to the extremes, it looks like this: find and produce great content and find the reader who would appreciate it most. You can create the best content in the world if your ideal readers don’t know about it – and equally, you can have a clear picture of who your audience are if you cannot produce or get your hands on the content they want.

ideal reader

Based on this, there are two big areas of publishing where AI can (and will) make an impact: content analysis, recommendation and creation; and audience analysis.

Audience analysis and engagement

This area of AI is already known to and used by many marketers at top companies, but this is something publishers could benefit from more. Actually, chances are that you are already using AI technologies in your marketing, you are just not realizing it !

While not everyone has access to powerful mechanisms that recommend just the perfect next show in Netflix (who would have thought that ‘period dramas featuring a strong female lead’ was even a thing, but apparently I’m into it), other tools are already available.

Chatbots

You probably want to personally talk to all your readers and fans – but, obviously, you can’t. (At least not to all of them.) Chatbots are not only here for better customer service and cats, but can be a powerful tool of audience engagement. AuthorBot is promising to be able to engage in ‘real’ conversations but most of them are still lacking the ‘second brain’. I tried the openly available WonderBot from the same company to talk about Alice in Wonderland with moderate success: it gives you information based on specific keyword, but not variants of it. While the technology is still developing and not all chatbots are as smart as Siri or Google Assistant, everyone can build one – even for free.

wonderbot

Analysis

If you would like to better understand your audience or better reach the potential audience who would be your ideal reader, you can do the following: analyse what kind of content they are visiting on your website, when and through what platforms  you can reach them the best and you can follow what they are talking about on social media (yes, creepy, I know).

Professional tools for this job include PeoplePattern: even their free online tool is more powerful than I could ever imagine. Just input a specific keyword (self-publishing), and wonder. This tool could be extremely powerful if you are narrowing down your niche market.

People Pattern tool

You can only use this kind of tools, if you know what you are looking for. Thankfully, there is a solution to analyse the trends within a specific audience. Manually looking up influencers to see what they are talking about and check for trending twitter hashtags is something machines can already do for us. While this kind of predictive deep analysis of customer behaviour is already existing and will definitely shape how publishers interact with readers in the future, these technologies are not yet available for smaller businesses or and authorpreneurs.

Advertising

Smart advertising, however, is available for everyone. Pay-per-click adverts only work if they target theideal reader:otherwise, they are only a waste of money. You will be relieved to know that Google AdWords is already using AI to show the most relevant ads to people, and will continue to improve its adaptability. (And on the other hand, as a customer yourself, you’ll probably see less and less from ‘creepy’ and irrelevant ads that come up for weeks after you’ve searched for something uncomfortable.)

Let’s move on to tools that are specifically designed to create, analyse and recommend content.

Content analysis, recommendation and creation

This area closely overlaps with the previous topic: chatbots are creating content, even if not very original one, and knowing the audience will result in better recommendation systems.

Content analysis

Analysing content (not only text but also video and audio) and letting AI do the job has the advantage of AI discovering hidden connections nobody could think of by themselves. But what makes two books similar? The style, however obscure it is? The characters? The setting? This is an area where data analysers will need the insight from librarians and publishers.

Content analysis is a great way to gain insights about what people are talking about, who is your competitor, to monetize your back catalogue or match agents with authors and books. What are the existing technologies and tools that are already available and accessible to step foot in a more intelligent publishing?

Google Cloud

Probably you already use cloud services such as Google Docs, Sheets or Photos on your Drive, but you probably have missed out some cool tools and services primarily targeted to developers provided by Microsoft, Amazon and Google. These services enable programmers to use some of the cutting edge technology already available and run their own apps on giant server, but many of them can be your next online toy too.

Why is this interesting for us? Because Google Cloud, in addition to providing great paid services to developers, has some low-cost and permanently free services with user friendly interface, so you can try them without any programming experience. (It actually offers all the services you need to run a small app!) Let’s see which are the most interesting ones for authors and publishers in the Google Cloud.

Vision

One of the free services Google offers is Vision: you can analyse 1000 images a month – for free. In addition to label detection and text detection (everybody is familiar with those who has ever needed to use Google Translate in camera mode), it filters for explicit content, faces, landmarks and properties.

This would help in advance to understand cultural differences such as the prudence of the US compared to European standards, not only realising it when your book is not accepted to iBooks with the error: “book cover contains explicit content”. And you are just looking at a renaissance painting with nude breasts…

While none of the above mention usages are surprising on their own (after a simple Google image search you can see relevant keywords and I assume all of us have our Google Photos automatically categorized by ‘pictures of churches’ and ‘pictures of ducks’), altogether they create a powerful tool that can be your first step when choosing or creating a book cover.

Feeding a book cover in the system, Google automatically detects that it is about Christmas (further down we can see ‘christmas decoration’ and ‘christmas bells’). Under the next tab, ‘Web’, Google does not only list important keywords that are associated with the image but a seemingly long and deep websites that are using the image, even if partially. Under the ‘Properties’ tab, there is a breakdown of the most dominant colours and meaningful cropping advice – I’m terrible at cropping, so this is something I can wholeheartedly welcome.

Vision tool

Not surprisingly, the API is much less talkative if we test it with a minimalist image. I would have been interested in learning about the designer, the copyright status or at least a ‘teenage boy in red hat’ would have been sufficient, but it seems like Google is failing us in this.

API

As a publisher, you can use this API to better monetize your back catalogue: just run a search on all your covers (you’ll need to be logged in to do this) and search for certain parameters.

Natural language

Another free Google tool that has future potential for writers and small publishers is Natural Language processing. According to the promise, the API is able to reveal the structure and meaning of any text by offering powerful machine learning models. It extracts information about people, places, events from any kind of text document. Expert users can use it to automatically recognize customer intent in support emails or use it as a first (or ‘zeroth’) reader.

While there is no character limit specified, the free version cannot work with book length texts (it won’t actually give an error message but state that ‘Service Unavailable: Please try again later’). It can nicely handle a book description though.

The colors mean different categories (eg. person, location, work of art), and the little numbers in the corner show when the system managed to match cross-references in the text. For example, it knows that Anne of Green Gablesis a novel and Lucy Maud Montgomery is the author,but it doesn’t know that Matthew and Marilla Cuthbert are brother and sister. It recognizes Montgomery’s later mentions as being about her, but later mentions of the word ‘novel’ do not match up with the first.

Natural language

For ease of use, everything comes nicely linked to Wikipedia.

Natural language-categories

The API also provides a sentiment analyser: is the text in general positive, neutral or negative? And what is the strength of the sentiments? The description of Anne is described as fairly neutral with low magnitude scores.

sentiment analyser

While I kind of agree with the results for Anne of Green Gables, the way the machine reads texts is still far from how humans do. For example, I couldn’t think of many things more positive than companies offering up all their profits for cancer research.

sentiment analyser pic

While this tool is already capable of providing keyword information and a general idea of how powerfully you convey feelings or emotions (or how neutral can you be if that’s required), publishers can already use the content categorization.

keyword

While in the long term it will hopefully be able to spit out BISAC codes, it can already find a subcategory within ‘Books & Literature’ (even though it is not confident about it).

Content recommendation

While some level of recommendation is already available in stores (content matching based on keywords and metadata, and ‘also bought’ recommendations using customer behaviour: Netflix and Amazon are leading the field on this. The keyword, as well as metadata based recommendation and prediction is flawed in a way that humans cannot think of each possible keyword and connection.

Current book recommendation platforms rely largely on metadata and other’s recommendations: if your friends, the critics or a famous person whose opinion you value have liked it, chances are that you will. Indeed there are other things to consider (not lastly for practical reasons: reading is solitary, but talking about books isn’t). But the problem many avid readers face: how do you discover new books? How do you discover that hidden gem that would be the logical next choice but nobody has ever heard of it?

From a reader’s perspective, Whichbook partially solves the problem. You set parameters (max. 4) and it creates you a reading list that fits them, using a powerful content analysis. The only backside is that the content has to be manually categorized and tagged by humans: it must be extremely time-consuming. But as we see the current status of AI – this is still the best method to give the most appropriate results within the statistical confidence interval.

whichbook tool

Interestingly enough, there have been extremely powerful recommendation tools available for readers: Booklamp, dubbed as Pandora for books, used to analyse story DNA and Oyster was the real Netflix for books: subscription service that knows what you should read next. It’s a shame that Booklamp was bought and shut down by Apple and Oyster by Google, their mechanisms probably getting utilised for the tech giants’ own services.

From the publisher’s perspective, knowing what books are most similar to yours (and who reads them) could help in better targeting and marketing for the future.

Content creation

It is not till long until a robot can write this article. One has already tried to write the next Game of Thrones and there is a code anyone can use to copy Rowlings’ or Follett’s style. However, this is still the age of the infinite monkey theorem. While there are some robot journalists, especially at the area of financial reports writing, they can only work according to a certain template: the journalists pre-write certain phrases according to different possible outcomes, then the bot analyses the data and matches the numbers to the correct chunks.

AI is not only able to answer customer queries but write your blog or come up with a great title.

Even when it is about translation we can see how AI may interfere the business as usual of the industry today. Google just released their version of Airpod with realtime translating option in 40 languages… Where translators already use services to transcript their spoken word, this innovation shows a more accessible future translation market by listening to or reading a book out loud and getting them translated realtime.

Future of AI and next challenges

Meanwhile some already are calling AI as the end of creative writing (which is clearly not here…), as an author you have to think about how you can be different if everyone can have Rowling’s or Follett’s style. And that is what machines cannot generate – though even Zuckerberg tried to build it.

The key lies in EQ or EI – whatever you call it – using emotional intelligence to engage your audience with the help of branding.

Building a Lovemark (Roberts, 2004) as your brand is not easy to copy by any machine and even more difficult to achieve. So as an authorpreneur you have to not just think about how to be a better creative writer but how to be a better storyteller for your author brand. All these AI tools can help you better understand your audience, your market, your competitors, but they cannot generate emotional bond between you and your readers. In order to be truly successful you have to leverage the power of the most intelligent tools to make better decisions and get closer to your readers. But without YOU, your personality, your values, your brand identity none of these tools can make an impact in your publishing process.

As the PubEx has put it: AI won’t take our jobs away but turn us into centaurs. Maybe the new era of robotized content creation will spark people’s imagination and inspire us to write better, more creative stories. What do you think? How will you and your business use AI in the future?

Please follow and like us:
0
  • Wow, I am simply amazed by the quality of your article, Zsofia: you should definitely write some books !!!
    That’s right: artificial intelligence will completely change our way of thinking, writing and… working. As “robots” will replace most of today’s jobs, it is hard not to be afraid by this radical change. But it happens for all technological revolutions: the luddites fought against modernity, and it will be the same for some of our fellow Earth’s citizens. Our world will be better with AI and “robots”. Decision-making will be wiser, and our “global conscience” (called “the internet”) will help us become better persons, I think.

    When I teach to my students today, trying to implement some “innovative” features into the old classroom learning style, and also in my publishing activity, I often stop and try to contemplate what is really happening under our eyes… And keep trying to think, over and over, that it will be for the better.

    Your article is full of information, of “leads” to follow… Yes, I would like to use the AI to reach the “right” readers even better, not to mimic other writers’ style of writing. AI will help people to reach the right knowledge at the right time. It is a heavy trend you can even see within blogs, within “infomercials” : people do not want BS anymore, they want “human”, they want “authenticity”. AI will help us focus on this essential aspect and help us sort and filter all the insignificant data produced by… robots.

    Thank you very much for your insights, Zsofia, you help us a lot.

    Kind regards,
    Olivier

  • Thank you Olivier! I’m glad you enjoyed the article. I really like the idea of how the internet will help us all become better people – for the sake of us all I wish that you were right.