Modeling Multimodal Expression of User's Affective Subjective Experience

Abstract

With the growing importance of information technology in our everydaylife, newtypesof applications are appearing that require the understanding of information in a broad sense.Information that includes affective and subjective content plays a major role not only in anindividual?s cognitive processes but also in an individual?s interaction with others.We identifythree key points to be considered when developing systems that capture affective information:embodiment (experiencing physical reality), dynamics (mapping experience and emotional statewith its label) and adaptive interaction (conveying emotive response, responding to a recognizedemotional state).We present two computational systems that implement those principles:MOUE(Model Of User Emotions) is an emotion recognition system that recognizes the user?s emotionfrom his/her facial expressions, and from it, adaptively builds semantic definitions of emotionconcepts using the user?s feedback; MIKE (Multimedia Interactive Environment for Kanseicommunication) is an interactive adaptive system that, along with the user, co-evolves a languagefor communicating over subjective impressions. With the growing importance of information technology in our everydaylife, newtypesof applications are appearing that require the understanding of information in a broad sense.Information that includes affective and subjective content plays a major role not only in anindividual?s cognitive processes but also in an individual?s interaction with others.We identifythree key points to be considered when developing systems that capture affective information:embodiment (experiencing physical reality), dynamics (mapping experience and emotional statewith its label) and adaptive interaction (conveying emotive response, responding to a recognizedemotional state).We present two computational systems that implement those principles:MOUE(Model Of User Emotions) is an emotion recognition system that recognizes the user?s emotionfrom his/her facial expressions, and from it, adaptively builds semantic definitions of emotionconcepts using the user?s feedback; MIKE (Multimedia Interactive Environment for Kanseicommunication) is an interactive adaptive system that, along with the user, co-evolves a languagefor communicating over subjective impressions.