Centrum för kognitiv semiotik (CCS)

Humanistiska och teologiska fakulteterna | Lunds universitet

Virtual Human Simulation of Awareness

Bilyana Martinovsky

Virtual Humans are computer simulations of human behavior. They are used for interactive learning, studying, training, negotiation, instruction, and entertainment. In order to interact with humans, Virtual Humans need to be able to process language and gestures in different contexts but they also need to process emotion. Emotion is defined here as a sensory feedback recognition and a coping strategy, which coordinate decision- making on individual level and in interaction (Cornelius 2000, Lazarus 1999). Given this definition, how can we simulate awareness of own and others’ cognitive-emotive states? These talk proposes a three level Mind-Minding model for development of awareness of motion, culture and emotion using iterative Interactive Alignment (Pickering&Garrod 2006) and Reciprocal Adaptation (Gumperz 1982) (Martinovski 2013).

Three competing Mind-Minding theories have been proposed to explain how humans understand each other’s states, namely Imitation Theory (Iacoboni 2005), Simulation Theory (Stich and Nichols 1989) and Representation Theory (Gordon 1986, Hobbs and Gordon 2004). They correspond to three types of semiotic relations: similarity, co-presence and convention. Discourse analysis of linguistic-pragmatic manifestations of icons, index and symbols indicates the usage of all three Mind-Minding processes in human interaction (Martinovsky 2007, 2015). Can all these be used in simulation of Virtual Human interaction? In this talk I will show two Virtual Human simulations. One illustrates an application of a Culture-based Interactive Motion Model. The second simulation is based on an Embedded Mind-Minding Cognitive Model. To simulate awareness of Self and Other (Buber 2000) in interaction, we need an Iterative Three-Level Mind-Minding Model of task, emotion and communication processing.

1.    Cornelius R.R. Theoretical approaches to emotion. Proceedings from ISCA Workshop on Speech and Emotion: A Conceptual Framework for Research (2000)
2.    Lazarus, R. Stress and Emotion. New York: Springer Publishing Company, Inc. (1999)
3.    Pickering, M. J. and Garrod S. Alignment as the basis for successful communication, Research on Language and Computation, Vol 4, pp 203-228 (2006)
4.    Gumperz, J.J. Discourse Strategies. Cambridge: Cambridge University Press (1982)
5.    Martinovsky, B. Reciprocal adaptation and problem reframing in group decision and negotiation. Journal of Group Decision and Negotiation. Vol. 23, No. 3 pp 497-
514. Springer (2013)
6.    Iacoboni, M. Understanding others: imitation, language, empathy. In S. Hurley and N. Chater (Eds) Perspectives on Imitation: From Cognitive Neuroscience to Social Science. Cambridge, MA: MIT Press (2005)
7.    Stich, S. and Nichols. S. Folk psychology: simulation or tacit theory? Mind and Language, 7, 35-71 (1992)
8.    Gordon, R. Folk psychology as simulation. Mind and Language, 1: 158-170 (1986)
9.    Hobbs, J. and Gordon, A. Encoding Knowledge of Commonsense Psychology. 7th International Symposium on Logical Formalizations of Commonsense Reasoning. May 22-24, 2005, Corfu, Greece (2005)
10.    Martinovsky, B. Empathy and theory of mind and body in evolution. In Ahlsén,
E. et al. (eds) Communication - Action - Meaning. A Festschrift to Jens Allwood. University of Gothenburg, Department of Linguistics, pp. 343-361 (2007)
11.    Martinovsky, B. (ed.) Emotion in Group  Decision  and  Negotiation. Amsterdam: Springer (2015)
12.    Buber, M. I and Thou. New York: Charles Scribner’s sons (2000)