|
The use of multimedia in education is increasingly prevalent. This post looks at the cognitive theory of multimedia learning, its sub-theories, and the use of video within the framework of Kalantzis' and Cope's seven principles for new learning and assessment as it relates to technology.
Why Video? Video and Multimodality 
Video is a form of technology that expresses multimodality. As Cope and Kalantzis point out in Making Sense - Reference, Agency, and Structure in a Grammar of Multimodal Meaning, “meanings are transposable across forms… but in the transposition, the meaning is never quite the same. Each form is partial. Its media have affordances, which offer both opportunities for meaning and constraints. This is why we need multimodality” (Cope and Kalantzis, 2020, p. 33). This is the reason human beings “habitually transpose meanings. Multimodal transposition is in our natures" (2020, p. 33).
Cognitive Theory of Multimedia Learning
Many studies have demonstrated the positive effectiveness of video as an educational tool (Brame, 2015), but the design of the content varies among its producers. Learning as the acquisition of knowledge that results in increased knowledge or changed behavior depends on memory. If memory consists of sensory memory, short-term memory, and long-term memory, their relationship and interaction should be taken into consideration when creating or selecting videos as tools for education – especially the capacity of a learner to capture and hold onto information in his/her memory. Richard Mayer is credited for the cognitive theory of multimedia learning. In addition to using the information processing model, he incorporated Paivio’s dual coding theory, Sweller’s cognitive load theory, and Baddeley’s model of working memory.
Information Processing Model
The information processing model of memory treats sensory memory as transient. Part of short-term memory’s function is to decide which data from our senses should be analyzed for processing. The information that is considered too trivial to keep is forgotten; the remainder is encoded and transferred to long-term memory. Short-term memory (working memory) is considered to have limits on how much information it can hold whereas long-term memory in comparison is limitless. George A Miller (1956) theorized that short term memory, can only hold 7 ± 2 bits of information “for about 20 seconds. Further, typical [working memory] can process (i.e., combine, contrast, or manipulate) about 2 to 4 elements of information” (Greer, Crutchfield, and Woods, 2013, p. 42). Atkinson and Shiffrin developed the information processing model in 1968 (Malmberg, Raaijmakers, and Shiffrin, 2019) and Figure 1 illustrates the journey incoming data makes through memory.
Figure 1
Information Processing Model 
Information Processing Model. “Adapted from Atkinson, R.C. and Shiffrin, R.M. (1968). 'Human memory: A Proposed System and its Control Processes'. In Spence, K.W. and Spence, J.T. The psychology of learning and motivation, (Volume 2). New York: Academic Press. pp. 89–195. (MindTools, n.d.) https://www.mindtools.com/pages/article/cognitive-load-theory.htm
Dual Coding Theory
Allan Paivio’s dual coding theory states that the dominant sensory modes are visual and auditory in terms of the way people receive information and that memory uses two separate channels to process the information coming from these senses. Visual information passes through the eyes, sound passes through the ears; the printed word passes through both the visual and auditory channels. Because printed words pass through both channels this has implications for when presenting complex tasks.
An example of applying dual coding theory is already seen in instruction of low-literacy ESL students. Words that are easily associated with images are taught first in accompaniment with the images that they represent When using the vocabulary in sentences, the students already have the images in their mind associated with the words. and would therefore be able to translate the words better this might be more effective than just having students use a translator to build vocabulary.
Image is from Swerdloff, M. (2016).
https://www.sciencedirect.com/topics/social-sciences/multimedia-learning Cognitive Load Theory
Cognitive load theory centers on working memory which is short term. It includes a discussion what interferes with learning as data is processed and analyzed in working memory. It states that because working memory has a limited capacity to hold information (Sweller, 1988), it performs a sort of triage on incoming data transmitted through sensory memory. It categorizes memory into three classes: intrinsic, germane and extraneous. Intrinsic information is that which is perceived to be relevant and vital to achieve the desired learning outcome. Germane information is relevant but not necessary. Extraneous information is irrelevant (though may be interesting) and will not achieve the desired learning outcome. If information is found to be extraneous to the learning objective, that is, it is not intrinsic nor germane to the desired learning outcome, it is disposed of (forgotten) and not sent to long term memory. The more extraneous data working memory is forced to analyze, the longer it takes for working memory to transfer relevant data to long term memory. This may result in learning activities having to be repeated before new knowledge can be considered acquired (Brame, 2015). Chunking information (into familiar bits of knowledge if possible) is recommended.
The following video (Bucy, 2009) provides an exercise that quickly demonstrates elements of cognitive load theory. Model of Working Memory
Richard Mayer also credits Alan Baddeley’s model for working memory (WM) as helping him develop his multimedia learning theory.
In 2000, Baddeley added another component to his model called Episodic buffer. Its role is to serve “as a 'backup' store which communicates with both long-term memory and the components of working memory” (MacLeod, 2012).
In an article issued in January 2020, Baddeley, Hitch and Allen explain the theory’s development since 2000. Mayer’s incorporation of the above theories did not stop those theories from evolving over time as has Mayer’s.
The image represents the revised multicomponent model of working memory. From “The Episodic Buffer: A New Component of Working Memory?” by A. D. Baddeley, . https://doi.org/10.3758/s13414-019-01837-x
Cognitive Theory of Multimedia Learning
For Mayer, “humans engage in active learning by attending to relevant incoming information, organizing selected information into coherent mental representations, and integrating mental representations with other knowledge” (Mayer, 2014, p. 47) See Table 1 and Figure 2.
Table 1.
The three assumptions of Mayer's cognitive theory of multimedia learning
(Mayer, 2014, p. 47) https://doi.org/10.1017/CBO9781139547369.005
Figure 2. Richard Mayer's Cognitive Theory of Multimedia Learning 
Image of Richard Mayer's Cognitive theory of multimedia learning model is from Learning-theories.org. https://www.learning-theories.org/doku.php?id=learning_theories:cognitive_theory_of_multimedia_learning (redrawn to improve sharpness)
The cognitive theory of multimedia learning states that humans intake visual information through one channel, and sound through a second; and that the written word is processed as both. The short-term/working memory processes the information and converts it to mental representations that incorporates what it can from prior knowledge stored in long term memory; however short term memory is limited in terms of the amount it can process, therefore, design of multimedia should take that into account. Humans construct meaningful knowledge when relevant material is selected, organized and integrated with prior learning.
In the video below, Rahul Patwari applies multimedia learning principles to a flipped classroom.
Patwari, R. (2015, April 9). Multimedia principles. [Video]. YouTube. https://www.youtube.com/watch?v=BcWSUnXz8kw
Patwari, besides discussing additional learning principles that come into play with multimedia learning, also discusses the need for chunking information. It is better to have information distributed through five short videos than one long one. So, how long should videos be? Brame (2015) reports that Guo et al examined the engagement levels of students based on video length from four MOOCs. Their results are from 6.9 million video-watching sessions. “They observed that the median engagement time for videos less than six minutes long was close to 100%– that is, students tended to watch the whole video (although there are significant outliers; see the paper for more complete information) (Brame, 2015). Other suggestions have stated that maximum length should be 10-12 minutes.
A study by Slemmons et al found that differences in immediate recall are negligible for long videos but there may be differences in ability to demonstrate understanding over a longer period of time depending on the gender and whether the student has a learning disability. While short-term retention of material did not seem to be influenced by video length, longer-term retention for males and students with learning disabilities was higher following short videos compared to long as assessed on summative assessments. Students self-report that they were more engaged, had enhanced focus, and had a perceived higher retention of content following shorter videos. This study has important implications for student learning, application of content, and the development of critical- thinking skills. This is particularly paramount in an era where content knowledge is just a search engine away. (Slemmons et al, 2018)
Applications for Video in the Classroom: The Seven Principles for New Learning and Assessment
Please note that this article was written earlier in the year before knowing how long the pandemic would last which is the reason the focus is on classroom applications; however, much of this applies to online learning as well.
The theories above have been centered primarily on memory and information processing. A general theory of technology by Kalantzis and Cope (n.d.) suggests how audio/video can be applied in a classroom.
(Kalantzis and Cope, n.d.) https://cgscholar.com/community/community_profiles/community-16395/community_updates/109824
Ubiquitous learning, active knowledge making, multimodal meaning, recursive feedback, collaborative intelligence, metacognition, and differentiated learning are the seven principles that are core elements of Kalantzis’ and Cope’s reflexive/ergative pedagogy. Reflexive/ergative theory sees the learner as an agent capable of producing knowledge and for this to occur, learning should allow for knowledge that is discoverable and navigational. Learning should also promote knowledge as judgment; knowledge acquired should be representable. Learning should take advantage of social, dialogical minds. Devices in service of learning, therefore, should be “cognitive protheses” (Kalantzis and Cope, 2018).
Table 2 looks at audio/video media’s capabilities in terms of the seven affordances.
Table 2.
Using Video in the Classroom and Reflexive/Ergative Pedagogy
Video has been part of education since televisions could be placed in a cart and rolled into classrooms. With the onset of the Internet and video sharing and hosting web sites, the use of this media has increased exponentially. The rise in online courses has jet-fueled the adoption of video especially in higher education. “Video has become an important part of higher education. It is integrated as part of traditional courses, serves as a cornerstone of many blended courses, and is often the main information delivery mechanism in MOOCs” (Brame, 2015, p. 1)
It is true that video have been the domain primarily of instructors in education (Kaltura, 2015), but there is potential in a student-centered environment, during middle school and high school years and in adult education to have students use video as a tool to demonstrate active knowledge making in multiple ways. Videography can be put to innovative use.
Innovation
Please note that this article was written earlier in the year before knowing how long the pandemic would last which is the reason this section is from an in-person teaching viewpoint, but it mention using interactive whiteboards.
When teachers and students think of recording video many think only of a video camera, the video option on their smart phone or laptop. They click the function on and record, but there other video technology exists that can be used for learning. 360-degree cameras can be used to create virtual reality and provide students an immersive experience. Teachers can use neck-mounted or head mounted cameras to film demonstrations from their point-of-view (IPOV – instructor point-of-view). Filming IPOV is easier to do when performing a task, but for instructors who want their students to see them work out a problem or write an example on whiteboard and talk about it as they do that is hard to video record. One could screen record writing on an interactive whiteboard and record the narration separately or an instructor could try using an innovative technology that comes from Northwestern University. It is called a lightboard and is also referred to as the learning glass. This technology allows teachers to write while facing the student. It was created by Michael Peshkin (Fung, 2018).
In the video below, Michael Peshkin, demonstrates the use of lightboard technology which is open source hardware.
Peshkin, M. (2013, June). Lightboard : a.k.a. learning glass. https://www.youtube.com/watch?v=N1I4Afti6XE
More information about about the open source hardware Lightboard (a.k.a. Learning glass) can be found at https://lightboard.info.
Evidence
Brame (2015) has reviewed multiple studies that have proven the effectiveness of video as an educational tool. Research has also discovered that novice learners benefit from auditory narration rather than reading words when they are also required to process other visual information (Greer, Crutchfield, and Woods, 2013). Carmichael, Reid, and Karpicke (n.d.) found that when an instructor was in the video, watching that instructor perform a task boosted students’ confidence in their ability to accomplish the same task. Research has also found that video-based learning can result in improvements in teaching (Carmichael, Reid, and Karpicke, n.d; Gainsburg, 2009; Seidel, Blomberg, and Renkl, 2013).
Criticisms
One of the earlier critiques of Mayer’s cognitive theory of multimedia learning is that it did not address motivation. Mayer (2014) reviews some of these studies. In his conclusion he states that “overall, the papers encourage us to consider instructional design features aimed at priming motivation to engage in deep processing during learning, while not overloading the learner’s information processing system” (2014). Another early critique targets long term memory, and its lack of attention compared to short-term memory in Mayer’s theory. In discussing long-term memory one has to bring constructivism. Constructivism talks about the assimilation/accommodation of prior knowledge when acquiring new knowledge. Multimedia learning theories have focused on what happens before information becomes part of long term memory a.k.a. prior knowledge. CTML does not ignore long-term memory. It views the relationship between short-term memory and long-term memory as dialogical.
Endel Tulvig (Harrell, 2020) does focus on long term memory and considers long-term memory to have channels. These channels are more like compartments. One for storing episodic events; another stores semantic information which Tulvig considers general knowledge. There is also separate storage for procedural information. Harrell (2020) mentions the story of Henry Molaison as evidence of this. Henry Molaison suffered a brain injury. Though he remembered how to do certain things, he was not able to remember the context within which he learned to do them.
Conclusion
The understanding of how a learner processes information is essential for those who use video as a multimodal learning aid. It is important that video presented for learning not overwhelm the brain’s capacity to process and retain information. The research and videos referred to and included in this work discuss decreasing the cognitive load by removing extraneous information and aligning texts near the relevant images discussed. Keeping the videos short can also help. Reducing the cognitive load makes it easier for students to process the relevant information and students find the use of video more engaging.
References
Brame, C.J. (2015). Effective educational videos. http://cft.vanderbilt.edu/guides-sub-pages/effective-educational-videos/
Bucy, M. (2009, October 18). Cognitive load exercise. [Video]. YouTube. https://www.youtube.com/watch?v=Rc705-WS2l4 Carmichael, M., Reid, A-K., Karpicke, J. (n.d.). Assessing the impact of educational video on student engagement critical thinking and learning: The current state of play. https://us.sagepub.com/sites/default/files/hevideolearning.pdf Cope, B., Kalantzis, M. (2020). Making sense: reference, agency, and structure in a grammar of multimodal meaning. United Kingdom: Cambridge University Press. https://www.google.com/books/edition/Making_Sense/0YDCDwAAQBAJ?hl=en&gbpv=0 Education at Illinois. (2016 ). E-Learning affordance 3a: Multimodal meaning. [Video]. YouTube. https://www.youtube.com/watch?v=S8fLr9CZg4o Fung, F. M. (2018, Jun 6). How innovative videography can supercharge education. https://theconversation.com/how-innovative-videography-can-supercharge-education-97676 Gainsburg, J. (2009). Creating effective video to promote student-centered teaching. Teacher Education Quarterly, 36(2), 163-178. https://www.jstor.org/stable/23479258?seq=1 Greer, D. L., Crutchfield, S. A., & Woods, K. L. (2013). Cognitive theory of multimedia learning, instructional design principles, and students with learning disabilities in computer-based and online learning environments. Journal of Education, 193(2), 41–50. https://doi.org/10.1177/002205741319300205 Harell, T. (2020, March 9). What is the modal model of memory? https://www.betterhelp.com/advice/memory/what-is-the-modal-model-of-memory/ Hitch, G.J., Allen, R.J. & Baddeley, A.D. (2020). Attention and binding in visual working memory: Two forms of attention and two kinds of buffer storage. Attention, Perception, & Psychophysics, 82, 280–293. https://doi.org/10.3758/s13414-019-01837-x Kalantzis, M. & Cope, B. (n.d.). An agenda for new learning and assessment: 7 principles. [Image]. [Course content]. In HRD 472: Learning technologies. University of Illinois Urbana-Champagne. https://cgscholar.com/community/community_profiles/community-16395/community_updates/109824 Kalantzis, M., & Cope, W. (2018). Multiliteracies: Meaning making and literacy learning in the era of digital text. Paper presented at University-Wide Teaching and Learning Symposium organized by Center of Teaching, Learning and Technology. https://ctlt.illinoisstate.edu/downloads/symposium/2018/Kalantzsis-Cope%20Morning.pdf Kaltura, Inc. (2015). The state of video in education 2015: A Kaltura report. http://site.kaltura.com/rs/984-SDM-859/images/The_State_of_Video_in_Education_2015_a_Kaltura_Report.pdf Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology, 107(3), 724–739. https://doi.org/10.1037/edu0000013 Learning-theories.org. (n.d.) [Richard Mayer’s cognitive theory of multimedia learning]. [Image]. https://www.learning-theories.org/doku.php?id=learning_theories:cognitive_theory_of_multimedia_learning McLeod, S. A. (2012). Working memory. Simply Psychology. https://www.simplypsychology.org/working%20memory.html Malmberg, K. J., Raaijmakers, J. G. W., and Shiffrin, R. M. (2019, January 28). 50 years of research sparked by Atkinson and Shiffrin: 1968. Memory & Cognition, 47, 561-574. https://doi.org/10.3758/s13421-019-00896-7 Mayer, R. (2014). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 43-71). Cambridge University Press. https://doi.org/10.1017/CBO9781139547369.005 Mayer, R. (2014). Incorporating motivation into multimedia learning. Learning and Instruction, 29, 171-173. https://shop.tarjomeplus.com/UploadFileEn/TPLUS_EN_4524.pdf Miller, G. (1956, March). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2) 81-97. https://pure.mpg.de/rest/items/item_2364276/component/file_2364275/content MindTools. (n.d.) Cognitive load theory: Helping people learn effectively. [Image]. https://www.mindtools.com/pages/article/cognitive-load-theory.htm Patwari, R. (2015, April 9). Multimedia principles. [Video]. YouTube. https://www.youtube.com/watch?v=BcWSUnXz8kw Peshkin, M. (2013, June). Lightboard : a.k.a. learning glass. [Video]. YouTube. https://www.youtube.com/watch?v=N1I4Afti6XE Schmidt, S. R. (2008). Cognitive psychology of memory. Learning and Memory: A Comprehensive Reference. https://www.sciencedirect.com/topics/neuroscience/dual-coding-theory Seidel, T., Blomberg, G., & Renkl, A. (2013). Instructional strategies for using video in teacher education. Teaching and Teacher Education, 34, 56-65. https://tinyurl.com/vezdmfd Slemmons, K., Anyanwu, K., Hames, J., Grabski, D., MLsna, J. Simkins, E., & Cook, P. (2018, May 16). The impact of video length on learning in a middle-level flipped science setting: Implications for diversity inclusion. https://doi.org/10.1007/s10956-018-9736-2 Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285 http://csjarchive.cogsci.rpi.edu/1988v12/i02/p0257p0285/MAIN.PDF Swerdloff, M. (2016) Online learning, multimedia, and emotions. Emotions, Technology, and Learning. https://www.sciencedirect.com/topics/social-sciences/multimedia-learning Comments are closed.
|
Upcoming Workshops
Be One of the 1sts to receive notifications about when new posts are uploaded.Interested in exploring digital online tools for teaching? Visit EPDI's sister site at https://www.adulted-technology.com/
Categories
All
Archives
April 2021
|
RSS Feed