Representing temporal concepts using redundant gestures in L2 ongoing interactions
Abstract
Human conversational interaction is multimodal, involving both verbal and non-verbal modalities. That is, when a speaker and listener interact, they use not only spoken messages but also manual gestures. Manual gestures and spoken messages are semantically and temporally related and work together to create and express a complete meaning. This study employs a data-driven approach to investigate how L2 learners spontaneously employ gestures to express temporal concepts in ongoing dyadic interactions using 11 recorded interactions among L2 learners. The distribution and frequency of specific types of manual gestures were examined using sequential and gesture analyses. The results showed that, when representing temporal concepts, the participants produced language-redundant gestures. For example, to convey temporal concepts, they tended to co-express the same information with manual gestures, namely abstract deictic and metaphoric gestures, on an imaginary mental timeline axis, which appeared to represent the English grammatical concepts of tense and aspectual meaning. Regarding the functional differences in gestures in interactions, based on sequential analysis, speakers employed language-redundant gestures to express time concepts explicitly in comprehension sequences and in the negotiation of meaning as a strategy of repair for lexical retrieval, paraphrasing, and clarification. These findings reveal that understanding the use of both modalities, speech and gestures, is critical in uncovering how speakers conceptualize time in their minds and integrate space and time in language.
Keywords:
Redundant gesture, multimodality, temporal concept, L2 interactionReferences
Alibali, M. W., Young, A. G., Crooks, N. M., Yeo, A., Wolfgram, M. S., Ledesma, I. M., Nathan, M. J., Church, R. B., and E. J. Knuth. 2013. Students learn more when their teacher has learned to gesture effectively. Gesture 13(2): 210–233. https://doi.org.10.1075/gest.13.2.05ali (Crossref)
Bavelas, J., Gerwing, J., Sutton, C., and D. Prevost. 2008. Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58(2): 495–520. https://doi.org.10.1016/j.jml.2007.02.004 (Crossref)
Casasanto, D., and K. Jasmin. 2012. The hands of time: Temporal gestures in English speakers. Cognitive Linguistics 23(4): 643–674. https://doi.org.10.1515/cog-2012-0020 (Crossref)
Cienki, A. 1998. Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J. P. Koenig (ed.), Discourse and cognition: Bridging the gap, 189–204. Stanford, CA: CSLI Publications.
de Ruiter, J. P., Bangerter, A., and P. Dings. 2012. The interplay between gesture and speech in the production of referring expressions: Investigating the tradeoff hypothesis. Topics in Cognitive Science 4(2): 232–248. https://doi.org.10.1111/j.1756-8765.2012.01183.x (Crossref)
Enfield, N. J., Kita, S., and J. P. de Ruiter. 2007. Primary and secondly pragmatic functions of pointing gestures. Journal of Pragmatics 39: 1722–1741. https://doi.org/10.1016/j.pragma.2007.03.001 (Crossref)
Goldin-Meadow, S. 2000. Beyond words: The importance of gesture to researchers and learners. Child Development 71(1): 231–239. http:doi.org/10.1111/1467-8624.00138 (Crossref)
Gregersen, T., Olivares-Cuhat, G., and J. Storm. 2009. An examination of L1 and L2 gesture use: What role does proficiency play? The Modern Language Journal 93(2): 195–208. https://doi.org.10.1111/j.1540-4781.2009.00856.x (Crossref)
Gullberg, M. 1998. Gesture as a communication strategy in second language discourse: A study of learners of French and Swedish. Lund: Lund University Press.
Hanamoto, H. 2020. How do co-occurring speech and gestures express meaning in a dyadic face-to-face ELF interaction? In R. A. Giri, A. Sharma, and J. D’Angelo (eds.), Functional variations in English: Theoretical considerations and practical challenges, 61–82. UK: Springer. https://doi.org/10.1007/978-3-030-52225-4_5 (Crossref)
Hanamoto, H. 2021. Improving spatial and temporal attention through the use of spontaneous gestures in dyadic English as lingua franca interactions. In C. Lee (ed.), Second language pragmatics and English language education in East Asia, 164–193. UK: Routledge. (Crossref)
Hostetter, A. B. 2011. When do gestures communicate? A met-analysis. Psychological Bulletin 137(2): 297–315. https://doi.org/10.1037/a0022128 (Crossref)
Jefferson, G. 1984. On the organization of laughter in talk about troubles. In J. M. Atkinson and J. Heritage (eds.), Structures of social action: Studies in conversation analysis, 346–369. Cambridge: Cambridge University Press. https://doi/org/10.1017/CBO9780511665868.021 (Crossref)
Kendon, A. 1980. Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (ed.), The relationship of verbal and nonverbal communication and language, 207–227. The Hague: Mouton De Gruyter Publishers. https://doi.org/10.1515/9783110813098.207 (Crossref)
Kendon, A. 1990. Spatial organization in social encounters: The F-formation system. In A. Kendon (ed.), Conducting interaction: Patterns of behavior in focused encounters, 209–237. Cambridge: Cambridge University Press.
Kendon, A. 1994. Do gestures communicate? A review. Research on Language and Social Interaction 27(3): 175–200. http://doi.org/10.1207/s15327973rlsi2703_2 (Crossref)
Kendon, A. 2004. Gesture: Visible action as utterance. New York: Cambridge University Press. http://doi.org/10.1017/CBO9780511807572 (Crossref)
Kita, S. 2000. How representational gestures help speaking. In D. McNeill (ed.), Language and gesture, 162–185. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511620850.011 (Crossref)
Kita, S., and A. Özyürek. 2003. What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48(1): 16–32. https://doi.org/10.1016/S0749-596X(02)00505-3 (Crossref)
Kita, S., Özyürek, A., Allen, S., Brown, A., Furman, R., and T. Ishizuka. 2007. Relations between syntactic encoding and co-speech gestures: Implications for a model of speech and gesture production. Language and Cognitive Processes 22(8): 1212–1236. https://doi.org/10.1080/01690960701461426 (Crossref)
Krauss, R. M., and U. Hadar. 1999. The role of speech-related arm/hand gestures in word retrieval. In L. S. Messing, and R. Campbell (eds.), Gesture, speech, and sign, 93–116. New York: Oxford University Press. https://doi.org/10.1093/acprof:oso/97801985524519.003.0006 (Crossref)
Krauss, R. M., Chen, Y., and R. F. Gottesman. 2000. Lexical gestures and lexical access: A process model. In D. McNeill (ed.), Language and gesture, 261–283. Cambridge, UK: Cambridge University Press. (Crossref)
Lakoff, G., and M. Johnson. 1980. Metaphors we live by. Chicago, IL: Chicago University Press.
McNeill, D. 1992. Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
McNeill, D. 2005. Gesture and thought. Chicago: University of Chicago Press. https://doi.org/10.7208/chicago/9780226514642.001.0001 (Crossref)
Meierkord, C. 2000. Interpreting successful lingua-franca interaction. An analysis of non-native/non-native small talk conversation in English. Linguistik Online 5: 1/00. https://doi.org/10.13092/lo.5.1013 (Crossref)
Morrel-Samuels, P., and R. M. Krauss. 1992. Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition 18(3): 615–622. https://doi.org/10.1037/0278-7393.18.3.615 (Crossref)
Schegloff, E. A. 1992. Repair after next turn: The last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology 97(5): 1295–1345. https://doi.org/10.1086/229903 (Crossref)
Sime, D. 2006. What do learners make of teachers’ gestures in the language classroom? International Review of Applied Linguistics in Language Teaching 44(2): 211– 230. https://doi.org/10.1515/IRAL2006.009 (Crossref)
So, W. C., Kita, S., and S. Goldin-Meadow. 2013. When do speakers use gestures to specify who does what to whom? The role of language proficiency and type of gestures in narratives. Journal of Psycholinguistic Research 42(6): 581–594. https//doi.org/10.1007/s10936-012-9230-6 (Crossref)
Torralbo, A., Santiago, J., and J. Lupiáñez. 2006. Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science 30(4): 745–757. https://doi.org/10.1207/s15516709cog0000_67 (Crossref)
Tuite, K. 1993. The production of gesture. Semiotica 93(1-2): 83–106. https://doi.org/10.1515/semi.1993.93.1-2.83 (Crossref)
Wagner, P., Malisz, Z., and S. Kopp. 2014. Gesture and speech in interaction: An overview. Speech Communication 57: 209–232. https://doi.org/10.1016/j.specom.2013.09.008 (Crossref)
Tokyo Denki University https://orcid.org/0000-0003-1811-4009
