What are natural language processing tools? A new study has analysed how they can be used to create social networks out of literary works including Game of Thrones.
The paper investigates how AI can become more “culturally aware”, and how natural language processing can be used in different types of texts.
In the study, the Vrije Universiteit Amsterdam and the Dutch Royal Academy’s Humanities Cluster evaluated four state-of-the-art tools for name recognition in texts. They did this to assess and improve performance on popular fiction, and found solutions which boost the ability of the tool to recognise names in one novel from seven percent accuracy up to ninety percent.
What is natural language processing?
Natural language processing is a subfield of artificial intelligence which is dedicated to the processing and analysis of natural language data by computers. Natural language processing (NLP) tools are commonly used in many day-to-day applications, for example Siri and Google.
Recognising names in texts
Natural language processing can be used to recognise names in texts. For example, a computer program can be fed newspaper articles in which humans have meticulously marked the names. The program can then learn what a name looks like based on the context or shape of the word, for example names often have capital letters or follow an honorific.
However, novelists have more freedom with naming than journalists do. Because some names in novels like Game of Thrones do not have ‘normal’ names, systems have difficulty recognising them.
Analysing novels as social networks
The paper states: “The analysis of literary works has experienced a surge in computer-assisted processing. To obtain insights into the community structures and social interactions portrayed in novels, the creation of social networks from novels has gained popularity…We find that there are no significant differences between old and modern novels but that both are subject to a large amount of variance.”
“We see this work as a step in creating more culturally-aware AI systems.”