Collaborative Multimodal Conversational Storytelling Systems

This research is concerned with the state of current interactive narrative and storytelling systems and how they can be applied to conversational systems of a multimodal collaborative nature.

Drawing on such diverse fields such as narratology, cybernetics, cognitive information retrieval, multimodal interaction, this research aims to investigate a model for how multimodal interface and conversational information systems can assist in the production of meaning and sense-making for collaborative storytelling applications.

Specifically, the basis of this model is derived from the cybernetician Gordon Pask, and his work on Conversation Theory (CT) (1975); the application of which is still being understood for its possible impact on education, information retrieval and knowledge management. Here the model is adapted and applied to a collaborative narrative and storytelling framework.

This research proposes a study of narrative intelligence and its relationship to the design of collaborative storytelling systems. Narrative intelligence seeks to understand how people assimilate experience into familiar narrative structures and create coherence, therefore it must influence the usability of collaborative storytelling systems.

With this assumption, the following questions will be addressed:

• Given the influence of our historical and cultural perspectives, what are the key design aspects of collaborative storytelling systems?

• In what way can the nature of multimodal interaction assist sense-making and narrative coherence in collaborative storytelling systems?

A key component of this research is the development and evaluation of an artifact, assimilate.net. This work is an online collaborative environment that allows participants to visually construct narratives in an abstract virtual space. The system is a visual search engine that aligns the participant’s search criteria with template stories drawn from a database of mythology and folklore. The narratives are presented as a symbolic language that consist of patterns of visual behaviour. These visual patterns are regulated and evaluated by the collective choices of participants, and by their semantic and rhetorical relationships to the other patterns in the system.

The assimilate system is controlled by a novel interface that creates the notion of a constructed ‘closed world’. It creates coherence and collective sense-making while glimpses of chaotic patterns that are semantically relevant may be seen outside. The system is in constant flux as meaning and connotation are cycled through a continuous process of search result feedback and narrative template selection.

With the adoption of a practice based methodology this research aims to make a significant contribution to collaborative storytelling systems of an emergent and generative nature.