Pushing boundaries in the measurement of language attitudes: Enhancing research practices with the L’ART Research Assistant app
Florian Breit
University College London & Bangor University , United Kingdomhttps://orcid.org/0000-0003-0976-4480
Marco Tamburelli
Bangor University , United Kingdomhttps://orcid.org/0000-0002-5472-0850
Ianto Gruffydd
Cardiff University & Bangor University , United Kingdomhttps://orcid.org/0009-0002-6242-9411
Lissander Brasca
Bangor University , United Kingdomhttps://orcid.org/0009-0002-6642-3682
Abstract
The importance of methodological developments has recently been emphasised both in language attitude research specifically (Kircher & Zipp 2022), and across linguistics and the social sciences more broadly, where there has been a particular focus on replicability (Sönnig & Werner 2021; Kobrock & Roettger 2023). One aspect of this concerns the adoption of more open, consistent, and comparable implementations of method.
We introduce a new digital application (the L’ART Research Assistant) for research in multilingualism and language attitudes. Designed specifically for work with populations speaking a majority and a regional/minority/minoritised/heritage language, the app implements reference versions of some common research methods and tasks. This benefits the research community by enhancing consistency and comparability within and across studies and by improving replicability and reproducibility.
We discuss technical and methodological considerations behind the app and illustrate its use with a brief case study of language attitudes across three European communities whose regional/minority languages receive radically different degrees of socio-political recognition: Lombard (Italy), Moselle-Franconian (Belgium), and Welsh (UK). The case study demonstrates not only how the app facilitates research across different communities that is easily comparable, results also reveal fundamental differences in attitude scores depending on the methods employed (AToL v. MGT). Consequently, we argue that there is a need to move toward both the adoption of more consistent, comparable methods as well as toward a more holistic approach to measuring language attitudes, where a battery of tests — as opposed to a single measure — should become the norm.
Keywords:
language attitudes, bilingualism, methodology, research tools, replicabilityReferences
Aguinis, H., C. Pierce, and S. Culpepper. 2009. Scale coarseness as a methodological artifact: Correcting correlation coefficients attenuated from using coarse scales. Organizational Research Methods 12(4): 623–652. https://doi.org/10.1177/1094428108318065 (Crossref)
Anderson, J. A., L. Mak, A. Keyvani Chahi, and E. Bialystok. 2016. The language and social background questionnaire: Assessing degree of bilingualism in a diverse population. Supplementary materials (Version 1). figshare. https://doi.org/10.6084/m9.figshare.3972486.v1
Anderson, J. A., L. Mak, A. Keyvani Chahi, and E. Bialystok. 2018. The language and social background questionnaire: Assessing degree of bilingualism in a diverse population. Behavior Research Methods 50(1): 250–263. https://doi.org/10.3758/s13428-017-0867-9 (Crossref)
Berez-Kroeker, A. L., L. Gawne, S. Smythe Kung, B. F. Kelly, T. Heston, G. Holton, P. Pulsifer, D. I. Beaver, S. Chelliah, S. Dubinsky, R. P. Meier, N. Thiberger, K. Rice, and A. C. Woodbury. 2019. Reproducible research in linguistics: A position statement on data citation and attribution in our field. Linguistics 56(1): 1–18. https://doi.org/10.1515/ling-2017-0032 (Crossref)
Brasca, L., M. Tamburelli, I. Gruffydd, and F. Breit. 2024. A matter of strength: Language policy, attitudes, and linguistic dominance in three bilingual communities. Journal of Multilingual and Multicultural Development. https://doi.org/10.1080/01434632.2024.2408448 (Crossref)
Breit, F., M. Tamburelli, I. Gruffydd, and L. Brasca. 2023. The L’ART Research Assistant: A digital toolkit for bilingualism and language attitude research [Manuscript]. Bangor University. https://doi.org/10.13140/RG.2.2.18801.35686
Chakroun, B. 2010. National qualification frameworks: From policy borrowing to policy learning. European Journal of Education 45(2): 199–216. https://doi.org/10.1111/j.1465-3435.2010.01425.x (Crossref)
Coluzzi, P. 2009. Endangered minority and regional languages (‘dialects’) in Italy. Modern Italy 14(1): 39–54. https://doi.org/10.1080/13532940802278546 (Crossref)
Coluzzi, P., L. Brasca, M. Trizzino, and S. Scuri. 2018. Language planning for Italian regional languages: The case of Lombard and Sicilian. In D. Stern, M. Nomachi, and B. Belić (eds.), Linguistic Regionalism in Eastern Europe and Beyond: Minority, Regional and Literary Microlanguages, 274–298. Peter Lang. https://doi.org/10.3726/b14464 (Crossref)
Diekmann, A. 2007. Empirische Sozialforschung: Grundlagen, Methoden, Anwendungen. Rowohlt.
Dragojevic, M., and S. Goatley-Soan. 2022. Americans’ attitudes towards foreign accents: evaluative hierarchies and underlying processes. Journal of Multilingual and Multicultural Development 43(2): 167–181. https://doi.org/10.1080/01434632.2020.1735402 (Crossref)
Echeverria, B. 2005. Language Attitudes in San Sebastian: The Basque Vernacular as Challenge to Spanish Language Hegemony. Journal of Multilingual and Multicultural Development 26: 249–264. https://doi.org/10.1080/01434630508668407 (Crossref)
Fishman, J. A. 2006. Language policy and language shift. In T. Ricento (ed.), An Introduction to Language Policy: Theory and Method, 311–328. Blackwell.
Grieve, J. 2021. Observation, experimentation, and replication in linguistics. Linguistics 59(5): 1342–1356. https://doi.org/10.1515/ling-2021-0094 (Crossref)
Jackson, P. W., and S. Messick. 1965. The person, the product, and the response: Conceptual problems in the assessment of creativity. Journal of Personality 33(3): 309–329. https://doi.org/10.1111/j.1467-6494.1965.tb01389.x (Crossref)
Kircher, R., and L. Zipp (eds.) 2022. Research methods in language attitudes. Cambridge University Press. https://doi.org/10.1017/9781108867788 (Crossref)
Kobrock, K., and T. B. Roettger. 2023. Assessing the replication landscape in experimental linguistics. Glossa Psycholinguistics 2(1): X. https://doi.org/10.5070/G6011135 (Crossref)
Lambert, W. E., R. C. Hodgson, R. C. Gardner, and S. Fillenbaum. 1960. Evaluational reactions to spoken language. Journal of Abnormal and Social Psychology 60(1): 44–51. https://doi.org/10.1037/h0044430 (Crossref)
Lehnert, T. E., S. Krolak-Schwerdt, and T. Hörstermann. 2016. Judging people and their language use: Evidence for the distinctness of attitudes towards languages and speakers’ nationality. Proceedings of EAPRIL 2015 2: 262–274.
Lehnert, T. E., S. Krolak-Schwerdt, and T. Hörstermann. 2018. Language and nationality attitudes as distinct factors that influence speaker evaluations: Explicit versus implicit attitudes in Luxembourg. Language & Communication 61: 58–70. https://doi.org/10.1016/j.langcom.2018.01.005 (Crossref)
Loureiro-Rodriguez, V., M. M. Boggess, and A. Goldsmith. 2013. Language attitudes in Galicia: Using the matched-guise test among high school students. Journal of Multilingual and Multicultural Development 34: 136–153. https://doi.org/10.1080/01434632.2012.729591 (Crossref)
Markel, N. N., R. M. Eisler, and H. W. Reese. 1967. Judging personality from dialect. Journal of Verbal Learning and Verbal Behavior 6(1): 33–35. https://doi.org/10.1016/S0022-5371(67)80044-6 (Crossref)
Moonesinghe, R., M. J. Khoury, and A. C. J. W. Janssens. (2007) Most Published Research Findings Are False—But a Little Replication Goes a Long Way. PLoS Medicine 4(2): e28. https://doi.org/10.1371/journal.pmed.0040028 (Crossref)
Pantos, A. 2019. Implicitness, automaticity, and consciousness in language attitudes research. Linguistics Vanguard 5(s1). https://doi.org/10.1515/lingvan-2018-0007 (Crossref)
Pharao, N., and T. Kristiansen. 2019. Reflections on the relation between direct/indirect methods and explicit/implicit attitudes. Linguistics Vanguard 5(s1). https://doi.org/10.1515/lingvan-2018-0010 (Crossref)
Price, A., and M. Tamburelli. 2020. Welsh-language prestige in adolescents: Attitudes in the heartlands. International Journal of Applied Linguistics 30(2): 195–213. https://doi.org/10.1111/ijal.12274 (Crossref)
Rehak, K. (2017, May 28). Are UUIDs really unique? Towards Data Science. https://towardsdatascience.com/are-uuids-really-unique-57eb80fc2a87
Russell, C. J., and P. Bobko. 1992. Moderated regression analysis and Likert scales: Too coarse for comfort. Journal of Applied Psychology 77(3): 336–342. https://doi.org/10.1037/0021-9010.77.3.336 (Crossref)
Salzberger, T., and M. Koller. 2019. The direction of the response scale matters — accounting for the unit of measurement. European Journal of Marketing 53(5): 871–891. https://doi.org/10.1108/EJM-08-2017-0539 (Crossref)
Schoel, C., J. Roessel, J. Eck, J. Janssen, B. Petrovic, A. Rothe, S. C. Rudert, and D. Stahlberg. 2013. ‘Attitudes Towards Languages’ (AToL) Scale: A Global Instrument. Journal of Language and Social Psychology 32(1): 21–45. https://doi.org/10.1177/0261927X12457922 (Crossref)
Simons, D. J. 2014. The value of direct replication. Perspectives on Psychological Science 9(1): 76–80. https://doi.org/10.1177/1745691613514755 (Crossref)
Sönnig, L., and V. Werner. (eds.) 2021. The replication crisis: Implications for linguistics [Special issue]. Linguistics 59(5). (Crossref)
Soukup, B. 2013. On matching speaker (dis)guises — revisiting a methodological tradition. In T. Kristiansen & S. Grondelaers (eds.), Language (des)standardisation in late modern Europe: Experimental studies, 267–285. Oslo: Novus.
Symonds, P. M. 1924. On the loss of reliability in ratings due to coarseness of the scale. Journal of Experimental Psychology 7(6): 456–461. https://doi.org/10.1037/h0074469 (Crossref)
Tarnate, J. 2022. Memory Game [Software]. CodePen. https://codepen.io/jeytii/pen/QoagLr
Vergnaud, A. C., M. Touvier, C. Méjean, E. Kesse-Guyot, C. Pollet, A. Malon, K. Castetbon, and S. Hercberg. 2011. Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Santé study. International Journal of Public Health 56(4): 407–417. https://doi.org/10.1007/s00038-011-0257-5 (Crossref)
Yan, T., T. Keusch, and L. He. 2018. The impact of question and scale characteristics on scale direction effects. Survey Practice 11(2). https://doi.org/10.29115/SP-2018-0008 (Crossref)
University College London & Bangor University https://orcid.org/0000-0003-0976-4480
