THE EFFECTS OF MULTIMODAL STORYTELLING ON COMPREHENSION, RECALL, AND DISCOURSE ENGAGEMENT IN PATIENTS WITH DEMENTIA

Abstract

Dementia causes a progressive decline in language comprehension and communication, limiting patients’ ability to engage in meaningful discourse. Although storytelling has been applied as a language-based intervention in dementia care, most existing approaches rely on single modes of presentation, such as verbal narration or visual cues, with little experimental evidence on the combined effect of multimodal communication. This pilot study investigated the effect of multimodal storytelling-integrating text, visuals, and gestures-on comprehension, recall, and discourse engagement in dementia patients. Guided by Multimodal Discourse Analysis (Kress & van Leeuwen, 2001) and cognitive theories of multimodal learning (Mayer, 2009), the study adopted a within-subjects quasi-experimental design involving six patients with mild to moderate dementia recruited from three tertiary hospitals. Each participant experienced three storytelling conditions: text-only, text plus visuals, and text plus visuals plus gestures. Short, age-appropriate narratives with clear sequences and concrete actions were used, delivered in 20–25 minute sessions twice weekly over four weeks. Storytelling was accompanied by illustrations and iconic gestures in the multimodal condition to support comprehension and engagement. Data were collected through structured comprehension tests, immediate and delayed recall tasks, observational engagement checklists, and audio/video recordings for multimodal discourse analysis. Quantitative analysis of comprehension and recall scores showed consistent increases across conditions, with the highest mean scores observed in the text + visuals
+gestures condition (comprehension mean = 9.67/11; delayed recall mean = 8.33/10). Observational data indicated greater attention, participation, and mirroring of gestures in the multimodal condition. Qualitative analysis of transcripts and recordings revealed improved narrative coherence, richer use of temporal and causal connectives, and active engagement with visual and gestural cues. The findings suggest that multimodal storytelling significantly enhanced comprehension, memory retention, and discourse production compared to unimodal approaches. The study provides preliminary, language-centered evidence supporting the integration of text, visuals, and gestures in dementia interventions and offers practical strategies for improving communication and interaction in clinical settings. These results lay the groundwork for larger-scale experimental research in applied linguistics and dementia care.