TY  - JOUR
A1  - Brunner, Marie-Louise
A1  - Diemer, Stefan
T1  - Multimodal meaning making: The annotation of nonverbal elements in multimodal corpus transcription
T2  - Research in Corpus Linguistics
N2  - The article discusses how to integrate annotation for nonverbal elements (NVE) from multimodal raw data as part of a standardized corpus transcription. We argue that it is essential to include multimodal elements when investigating conversational data, and that in order to integrate these elements, a structured approach to complex multimodal data is needed. We discuss how to formulate a structured corpus-suitable standard syntax and taxonomy for nonverbal features such as gesture, facial expressions, and physical stance, and how to integrate it in a corpus. Using corpus examples, the article describes the development of a robust annotation system for spoken language in the corpus of Video-mediated English as a Lingua Franca Conversations (ViMELF 2018) and illustrates how the system can be used for the study of spoken discourse. The system takes into account previous research on multimodality, transcribes salient nonverbal features in a concise manner, and uses a standard syntax. While such an approach introduces a degree of subjectivity through the criteria of salience and conciseness, the system also offers considerable advantages: it is versatile and adaptable, flexible enough to work with a wide range of multimodal data, and it allows both quantitative and qualitative research on the pragmatics of interaction.
KW  - corpus annotation
KW  - corpus transcription
KW  - multimodality
KW  - nonverbal elements
KW  - spoken discourse
KW  - video-mediated communication
KW  - gestures
KW  - Korpus <Linguistik>
KW  - Transkription
KW  - Annotation
KW  - Multimodalität
KW  - Mündliche Kommunikation
KW  - Nichtverbale Kommunikation
KW  - Visuelle Kommunikation
KW  - Geste
Y1  - 2021
UR  - https://hst.opus.hbz-nrw.de/frontdoor/index/index/docId/256
UR  - https://nbn-resolving.org/urn:nbn:de:hbz:tr5-2561
VL  - 9
IS  - 1
SP  - 63
EP  - 88
PB  - Asociación Española de Lingüística de Corpus, Spain
ER  -