讲座题目：Multimodal Analysis of Video-based process Data
讲 座 人：郑东萍 教授（夏威夷大学）
时 间：2019年6月24日（周一）15：00－17：00
地 点：北京外国语大学 东院图书馆三层报告厅
讲座人简介：
郑东萍博士现任教于美国夏威夷大学二语研究系，博士生导师。美国康涅狄克州大学教育心理学博士。国际互动关系、语言及认知学会常任理事，广东外语外贸大学外国语言学及应用语言学研究中心兼职研究员，吉林外国语大学白山学者。学术论文发表于世界著名的语言与跨文化交流SCI 、SSCI期刊，如Modern Language Journal, Language Sciences, Linguistics and Education等。
课题研究包括：1. 怎样通过设计虚拟仿真游戏空间环境扩展其跨文化交流与协调的功能性，及跨文化双向互动对语言认知和文化交流技巧的影响；2. 用移动无限通讯技术为平台的设计App，检测并培养对地方处所环境及他人的觉知性，从而让学习者达到即时的学以致用，扩展认知、学习和文化交流空间，激发社会文化参与和实现价值的潜能；3. 创新立足于国学、适于亚洲文化社会的语言学习理论与实践。主要提出行动协商理论(Negotiation for Action)、言语行动的价值实现理论(Languaging as Values-realizing) 、关护行动语言学习理论 (Language Learners as Caretakers)、从语言学习者到跨文化场所体验者（From Language Learner’s identity to Cross-Cultural and Place-Taking Persons)。
研究领域为：应用生态心理学，对话学，分布认知及语言理论来扩张以人文为中心的传统二语习得理论；基于儒家的角色伦理学和道德经发展，诠释与发展以友谊（event friendship)、集体分布（distributed)、言语行动（languaging)为中心的寓教于乐的学习理论，以乐感文化和正念为驱动的跨文化交流行动。
讲座内容：
With the increasing prominence of digitally-mediated communication, it has grown increasingly simple to incorporate a variety of modes, such as speech, writing, and visual images, into a single text. This increased potential for multimodality has called for a shift in focus from any one single mode to a framework which can incorporate all modes equally. Kress (2003) argues for an approach based on semiotic analysis, treating all elements of a text as signs which are purposefully chosen by and reflect the interests, perspectives, positions, and values of the sign-maker. Thus, the concept of literacy becomes not just the use of the resources of writing, but redefined through the broader scope of multimodal literacy in which the making and interpreting of signs is constrained by the affordances of the modes and resources involved, and simultaneously transforms both the semiotic resources at hand and the inner dispositions of those performing the meaning-making. The multimodal world of communication, Kress argues, now calls not simply for acquisition or competence but for design, the creative agency of the sign-maker to assess the affordances of the situation and incorporate their own interests and desires in the creation of new signs.
Kress’s approach demonstrates an increasing relevance for an ecological, dialogical and distributed (EDD) thinking in research on multimodality and new literacies, treating communication and interaction as ecological, agentive, and holistic. From an EDD perspective, cognition and learning are embodied, situated, dynamic and values-realizing. In tracing the development of EDD views applied to second language (L2) learning with technologies that afford avatar embodiment and place-based interactivity, this talk first will review four trends of multimodal analysis traditions (work done in collaboration with Kristyn Martin). Integrating EDD and selected analytical framework, this talk promotes an action-oriented sociocultural orientation to cognition and L2 learning. Such holistic learning requires new research methods to reveal how learning occurs and new learning results. Such results may include cultivated team relationships, friendships, coordination skills, cultural awareness. Multimodal analysis is one of the inclusive methods that can partially answer new research questions under the EDD views.
While Kress and many approaches to multimodality focus on the historical trends of literacy and the increasingly multimodal format of representation and communication, distributed cognition can offer a deeper look at the dynamic action-based processes involved in multimodal sense-making. In this paper, we will briefly review the trends of multimodality and new literacy research, and discuss how distributed cognition can bring the concepts of ecological values-realizing and the distributed cognitive system into the picture for more holistic analytical and pedagogical approach. Using multimodal data from student interactions in online virtual worlds and from Japanese television shows, we will demonstrate the need for analytical and pedagogical methods which are integrative and account for the agency and creativity of the sense-maker, particularly in the field of applied linguistics. We will then provide brief example of the application of the distributed cognitive view to multimodal analysis, and close with a discussion of how this integration of analytic approaches might be developed into a more holistic framework.
中国外语与教育研究中心
2019年6月5日
