PROFESSIONAL DEVELOPMENT FOR TEACHERS, TUTORS AND TRAINERS
  • Home
  • All Blog Posts
  • Courses
    • Upcoming
    • Create a multimodal Frayer model
  • About
  • Contact
  • Home
  • All Blog Posts
  • Courses
    • Upcoming
    • Create a multimodal Frayer model
  • About
  • Contact
Search

Video as a Learning Aid and the Cognitive Theory of Multimedia Learning

12/21/2020

 
​The use of multimedia in education is increasingly prevalent. This post looks at the cognitive theory of multimedia learning, its sub-theories, and the use of video within the framework of Kalantzis' and Cope's seven principles for new learning and assessment as it relates to technology. 

​Why Video? Video and Multimodality ​

Picture
Video is a form of technology that expresses multimodality. As Cope and Kalantzis point out in Making Sense - Reference, Agency, and Structure in a Grammar of Multimodal Meaning, “meanings are transposable across forms… but in the transposition, the meaning is never quite the same. Each form is partial. Its media have affordances, which offer both opportunities for meaning and constraints. This is why we need multimodality” (Cope and Kalantzis, 2020, p. 33). This is the reason human beings “habitually transpose meanings. Multimodal transposition is in our natures" (2020, p. 33). 

​Cognitive Theory of Multimedia Learning 

Many studies have demonstrated the positive effectiveness of video as an educational tool (Brame, 2015), but the design of the content varies among its producers. Learning as the acquisition of knowledge that results in increased knowledge or changed behavior depends on memory. If memory consists of sensory memory, short-term memory, and long-term memory, their relationship and interaction should be taken into consideration when creating or selecting videos as tools for education – especially the capacity of a learner to capture and hold onto information in his/her memory. Richard Mayer is credited for the cognitive theory of multimedia learning. In addition to using the information processing model, he incorporated Paivio’s dual coding theory, Sweller’s cognitive load theory, and Baddeley’s model of working memory. 

Information Processing Model

​The information processing model of memory treats sensory memory as transient. Part of short-term memory’s function is to decide which data from our senses should be analyzed for processing. The information that is considered too trivial to keep is forgotten; the remainder is encoded and transferred to long-term memory. Short-term memory (working memory) is considered to have limits on how much information it can hold whereas long-term memory in comparison is limitless. George A Miller (1956) theorized that short term memory, can only hold 7 ± 2 bits of information “for about 20 seconds. Further, typical [working memory] can process (i.e., combine, contrast, or manipulate) about 2 to 4 elements of information” (Greer, Crutchfield, and Woods, 2013, p. 42). Atkinson and Shiffrin developed the information processing model in 1968 (Malmberg, Raaijmakers, and Shiffrin, 2019) and Figure 1 illustrates the journey incoming data makes through memory. 
Figure 1

​Information Processing Model
Picture
Information Processing Model. “Adapted from Atkinson, R.C. and Shiffrin, R.M. (1968). 'Human memory: A Proposed System and its Control Processes'. In Spence, K.W. and Spence, J.T. The psychology of learning and motivation, (Volume 2). New York: Academic Press. pp. 89–195. (MindTools, n.d.) https://www.mindtools.com/pages/article/cognitive-load-theory.htm

Dual Coding Theory

​Allan Paivio’s dual coding theory states that the dominant sensory modes are visual and auditory in terms of the way people receive information and that memory uses two separate channels to process the information coming from these senses. Visual information passes through the eyes, sound passes through the ears; the printed word passes through both the visual and auditory channels. Because printed words pass through both channels this has implications for when presenting complex tasks. 
​Concrete language is remembered better than abstract language in a wide variety of tasks” (Paivio et al., 1994, p. 1196). The concreteness effect is most often interpreted within Paivio’s (1971) dual-coding theory, according to which imaginal and verbal processing independently contribute to memory for concrete words, whereas only verbal processing is usually possible for abstract material (Schmidt, 2008). ​
An example of applying dual coding theory is already seen in instruction of low-literacy ESL students. Words that are easily associated with images are taught first in accompaniment with the images that they represent When using the vocabulary in sentences, the students already have the images in their mind associated with the words. and would therefore be able to translate the words better this might be more effective than just having students use a translator to build vocabulary. 
Picture
Image is from Swerdloff, M. (2016).
https://www.sciencedirect.com/topics/social-sciences/multimedia-learning 

Cognitive Load Theory

​Cognitive load theory centers on working memory which is short term. It includes a discussion what interferes with learning as data is processed and analyzed in working memory. It states that because working memory has a limited capacity to hold information (Sweller, 1988), it performs a sort of triage on incoming data transmitted through sensory memory. It categorizes memory into three classes: intrinsic, germane and extraneous. Intrinsic information is that which is perceived to be relevant and vital to achieve the desired learning outcome. Germane information is relevant but not necessary. Extraneous information is irrelevant (though may be interesting) and will not achieve the desired learning outcome. If information is found to be extraneous to the learning objective, that is, it is not intrinsic nor germane to the desired learning outcome, it is disposed of (forgotten) and not sent to long term memory. The more extraneous data working memory is forced to analyze, the longer it takes for working memory to transfer relevant data to long term memory. This may result in learning activities having to be repeated before new knowledge can be considered acquired (Brame, 2015). Chunking information (into familiar bits of knowledge if possible) is recommended. 

​The following video (Bucy, 2009) provides an exercise that quickly demonstrates elements of cognitive load theory. 

Model of Working Memory

​Richard Mayer also credits Alan Baddeley’s model for working memory (WM) as helping him develop his multimedia learning theory. 
Baddeley described three mechanisms within the WM framework that relate to these WM limitations: a central executive (CE) that coordinates the WM processes and two subsystems, a visual/spatial processor, and a phonological loop or auditory processor (Sweller, 2005).The CE has four main tasks: a) focus attention; b) divide attention between more than one task as necessary, c) switch attention back and forth, and d) communicate with long term memory (Baddeley, 2012).The visual/spatial sketchpad stores visual and spatial information and is also responsible for creating and interpreting mental images (Swanson & Saez, 2005).The phonological loop acts as a temporary storage facility for verbal information, including subvocal speech and sounds. These three WM mechanisms interact with incoming information” (Greer, Crutchfield, and Woods, 2013). ​
In 2000, Baddeley added another component to his model called Episodic buffer. Its role is to serve “as a 'backup' store which communicates with both long-term memory and the components of working memory” (MacLeod, 2012). 
In an article issued in January 2020, Baddeley, Hitch and Allen explain the theory’s development since 2000. Mayer’s incorporation of the above theories did not stop those theories from evolving over time as has Mayer’s.
Picture
The image represents the revised multicomponent model of working memory. From “The Episodic Buffer: A New Component of Working Memory?” by A. D. Baddeley, . https://doi.org/10.3758/s13414-019-01837-x

Cognitive Theory of Multimedia Learning

For Mayer, “humans engage in active learning by attending to relevant incoming information, organizing selected information into coherent mental representations, and integrating mental representations with other knowledge” (Mayer, 2014, p. 47) See Table 1 and Figure 2. 
​Table 1. 

The three assumptions of Mayer's cognitive theory of multimedia learning 
Assumption Description
Dual channels Humans possess separate channels for processing visual and auditory information
Limited capacity Humans are limited in the amount of information that can be processed in each channela t one time
Active Processing Humans engage in active learning by attending to relevant incoming information, organizing selected information into coherent mental representations, and integrating mental representations with other knowledge
(Mayer, 2014, p. 47) https://doi.org/10.1017/CBO9781139547369.005

​Figure 2. 

Richard Mayer's Cognitive Theory of Multimedia Learning
Picture
​

Image of Richard Mayer's Cognitive theory of multimedia learning model is from Learning-theories.org. https://www.learning-theories.org/doku.php?id=learning_theories:cognitive_theory_of_multimedia_learning (redrawn to improve sharpness)
The cognitive theory of multimedia learning states that humans intake visual information through one channel, and sound through a second; and that the written word is processed as both. The short-term/working memory processes the information and converts it to mental representations that incorporates what it can from prior knowledge stored in long term memory; however short term memory is limited in terms of the amount it can process, therefore, design of multimedia should take that into account. Humans construct meaningful knowledge when relevant material is selected, organized and integrated with prior learning.
​
In the video below, Rahul Patwari applies multimedia learning principles to a flipped classroom.
Patwari, R. (2015, April 9). Multimedia principles. [Video]. YouTube. https://www.youtube.com/watch?v=BcWSUnXz8kw

​Patwari, besides discussing additional learning principles that come into play with multimedia learning, also discusses the need for chunking information. It is better to have information distributed through five short videos than one long one. So, how long should videos be? Brame (2015) reports that Guo et al examined the engagement levels of students based on video length from four MOOCs. Their results are from 6.9 million video-watching sessions. “They observed that the median engagement time for videos less than six minutes long was close to 100%– that is, students tended to watch the whole video (although there are significant outliers; see the paper for more complete information) (Brame, 2015). Other suggestions have stated that maximum length should be 10-12 minutes.
A study by Slemmons et al found that differences in immediate recall are negligible for long videos but there may be differences in ability to demonstrate understanding over a longer period of time depending on the gender and whether the student has a learning disability. While short-term retention of material did not seem to be influenced by video length, longer-term retention for males and students with learning disabilities was higher following short videos compared to long as assessed on summative assessments. Students self-report that they were more engaged, had enhanced focus, and had a perceived higher retention of content following shorter videos. This study has important implications for student learning, application of content, and the development of critical- thinking skills. This is particularly paramount in an era where content knowledge is just a search engine away. (Slemmons et al, 2018)

Applications for Video in the Classroom: The Seven Principles for New Learning and Assessment

Please note that this article was written earlier in the year before knowing how long the pandemic would last which is the reason the focus is on classroom applications; however, much of this applies to online learning as well.
The theories above have been centered primarily on memory and information processing. A general theory of technology by Kalantzis and Cope (n.d.) suggests how audio/video can be applied in a classroom.
Picture
(Kalantzis and Cope, n.d.) https://cgscholar.com/community/community_profiles/community-16395/community_updates/109824
Ubiquitous learning, active knowledge making, multimodal meaning, recursive feedback, collaborative intelligence, metacognition, and differentiated learning are the seven principles that are core elements of Kalantzis’ and Cope’s reflexive/ergative pedagogy. Reflexive/ergative theory sees the learner as an agent capable of producing knowledge and for this to occur, learning should allow for knowledge that is discoverable and navigational. Learning should also promote knowledge as judgment; knowledge acquired should be representable. Learning should take advantage of social, dialogical minds. Devices in service of learning, therefore, should be “cognitive protheses” (Kalantzis and Cope, 2018).

​Table 2 looks at audio/video media’s capabilities in terms of the seven affordances.
Table 2.

​Using Video in the Classroom and Reflexive/Ergative Pedagogy

The 7 principles of Technology 

Benefit 

Application Example  

ubiquitous learning 

video is accessible through smartphones, iPads, laptops, desktops 

videos can used for flipped classrooms 

active knowledge making 

learners can create their own videos to demonstrate knowledge 

videos can be created from the students’ point of view 

multimodal meaning 

video allows knowledge production using visual, verbal and auditory modes or representation that occur at the same time 

students can include video in Word documents that enhance multimodal expressions 

recursive feedback 

video recordings can be used as substitution to submit regular feedback 

videos can be used by teachers to provide feedback and for scaffolding if the teacher films themselves demonstrating performing a task 

collaborative intelligence 

learners can work together to produce a video 

The production of a video could be an interdisciplinary collaboration 

metacognition 

learners can create video recordings of their reflections-in-action/ reflections-on-action/judgement on learning 

videos can be used as a substitute for the typed word for student self-assessment in the form of self-reflections 

differentiated instruction 

video media can be used as an option in differentiated instruction when it comes to product and process. 

video can be used as part of presenting difficult concepts in multiple ways – lecture plus video and/or as option as to how students can demonstrate their knowledge acquisition. 


​Video has been part of education since televisions could be placed in a cart and rolled into classrooms. With the onset of the Internet and video sharing and hosting web sites, the use of this media has increased exponentially. The rise in online courses has jet-fueled the adoption of video especially in higher education. “Video has become an important part of higher education. It is integrated as part of traditional courses, serves as a cornerstone of many blended courses, and is often the main information delivery mechanism in MOOCs” (Brame, 2015, p. 1)
The majority of these online courses rely on audiovisual instruction supplemented by regular individual or collaborative homework assignments. The format of audiovisual instruction widely varies across courses, but one popular template is a series of roughly 12-min lecture videos showing lecture slides, overlaid with annotation and the instructor’s voice. (Kizilcec, Bailenson, and Gomez, 2015).
​It is true that video have been the domain primarily of instructors in education (Kaltura, 2015), but there is potential in a student-centered environment, during middle school and high school years and in adult education to have students use video as a tool to demonstrate active knowledge making in multiple ways. Videography can be put to innovative use.

Innovation

Please note that this article was written earlier in the year before knowing how long the pandemic would last which is the reason this section is from an in-person teaching viewpoint, but it mention using interactive whiteboards.
When teachers and students think of recording video many think only of a video camera, the video option on their smart phone or laptop. They click the function on and record, but there other video technology exists that can be used for learning. 360-degree cameras can be used to create virtual reality and provide students an immersive experience. Teachers can use neck-mounted or head mounted cameras to film demonstrations from their point-of-view (IPOV – instructor point-of-view). Filming IPOV is easier to do when performing a task, but for instructors who want their students to see them work out a problem or write an example on whiteboard and talk about it as they do that is hard to video record. One could screen record writing on an interactive whiteboard and record the narration separately or an instructor could try using an innovative technology that comes from Northwestern University. It is called a lightboard and is also referred to as the learning glass. This technology allows teachers to write while facing the student. It was created by Michael Peshkin (Fung, 2018).

​In the video below, Michael Peshkin, demonstrates the use of lightboard technology which is open source hardware.
Peshkin, M. (2013, June). Lightboard : a.k.a. learning glass. https://www.youtube.com/watch?v=N1I4Afti6XE
More information about about the open source hardware Lightboard (a.k.a. Learning glass) can be found at https://lightboard.info.

Evidence

Brame (2015) has reviewed multiple studies that have proven the effectiveness of video as an educational tool.  ​Research has also discovered that novice learners benefit from auditory narration rather than reading words when they are also required to process other visual information (Greer, Crutchfield, and Woods, 2013). Carmichael, Reid, and Karpicke (n.d.) found that when an instructor was in the video, watching that instructor perform a task boosted students’ confidence in their ability to accomplish the same task. Research has also found that video-based learning can result in improvements in teaching (Carmichael, Reid, and Karpicke, n.d; Gainsburg, 2009; Seidel, Blomberg, and Renkl, 2013).

Criticisms

​One of the earlier critiques of Mayer’s cognitive theory of multimedia learning is that it did not address motivation. Mayer (2014) reviews some of these studies. In his conclusion he states that “overall, the papers encourage us to consider instructional design features aimed at priming motivation to engage in deep processing during learning, while not overloading the learner’s information processing system” (2014). Another early critique targets long term memory, and its lack of attention compared to short-term memory in Mayer’s theory. In discussing long-term memory one has to bring constructivism. Constructivism talks about the assimilation/accommodation of prior knowledge when acquiring new knowledge. Multimedia learning theories have focused on what happens before information becomes part of long term memory a.k.a. prior knowledge. CTML does not ignore long-term memory. It views the relationship between short-term memory and long-term memory as dialogical.
Endel Tulvig (Harrell, 2020) does focus on long term memory and considers long-term memory to have channels. These channels are more like compartments. One for storing episodic events; another stores semantic information which Tulvig considers general knowledge. There is also separate storage for procedural information. Harrell (2020) mentions the story of Henry Molaison as evidence of this. Henry Molaison suffered a brain injury. Though he remembered how to do certain things, he was not able to remember the context within which he learned to do them.

Conclusion

​The understanding of how a learner processes information is essential for those who use video as a multimodal learning aid. It is important that video presented for learning not overwhelm the brain’s capacity to process and retain information. The research and videos referred to and included in this work discuss decreasing the cognitive load by removing extraneous information and aligning texts near the relevant images discussed. Keeping the videos short can also help. Reducing the cognitive load makes it easier for students to process the relevant information and students find the use of video more engaging.

References

​Brame, C.J. (2015). Effective educational videos. http://cft.vanderbilt.edu/guides-sub-pages/effective-educational-videos/

Bucy, M. (2009, October 18). Cognitive load exercise. [Video]. YouTube. https://www.youtube.com/watch?v=Rc705-WS2l4

Carmichael, M., Reid, A-K., Karpicke, J. (n.d.). Assessing the impact of educational video on student engagement critical thinking and learning: The current state of play. https://us.sagepub.com/sites/default/files/hevideolearning.pdf

Cope, B., Kalantzis, M. (2020). Making sense: reference, agency, and structure in a grammar of multimodal meaning. United Kingdom: Cambridge University Press. https://www.google.com/books/edition/Making_Sense/0YDCDwAAQBAJ?hl=en&gbpv=0

Education at Illinois. (2016 ). E-Learning affordance 3a: Multimodal meaning. [Video]. YouTube. https://www.youtube.com/watch?v=S8fLr9CZg4o

Fung, F. M. (2018, Jun 6). How innovative videography can supercharge education. https://theconversation.com/how-innovative-videography-can-supercharge-education-97676

Gainsburg, J. (2009). Creating effective video to promote student-centered teaching. Teacher Education Quarterly, 36(2), 163-178. https://www.jstor.org/stable/23479258?seq=1

Greer, D. L., Crutchfield, S. A., & Woods, K. L. (2013). Cognitive theory of multimedia learning, instructional design principles, and students with learning disabilities in computer-based and online learning environments. Journal of Education, 193(2), 41–50. https://doi.org/10.1177/002205741319300205

Harell, T. (2020, March 9). What is the modal model of memory? https://www.betterhelp.com/advice/memory/what-is-the-modal-model-of-memory/

Hitch, G.J., Allen, R.J. & Baddeley, A.D. (2020). Attention and binding in visual working memory: Two forms of attention and two kinds of buffer storage. Attention, Perception, & Psychophysics, 82, 280–293. https://doi.org/10.3758/s13414-019-01837-x

Kalantzis, M. & Cope, B. (n.d.). An agenda for new learning and assessment: 7 principles. [Image]. [Course content]. In HRD 472: Learning technologies. University of Illinois Urbana-Champagne. https://cgscholar.com/community/community_profiles/community-16395/community_updates/109824

Kalantzis, M., & Cope, W. (2018). Multiliteracies: Meaning making and literacy learning in the era of digital text. Paper presented at University-Wide Teaching and Learning Symposium organized by Center of Teaching, Learning and Technology. https://ctlt.illinoisstate.edu/downloads/symposium/2018/Kalantzsis-Cope%20Morning.pdf

Kaltura, Inc. (2015). The state of video in education 2015: A Kaltura report. http://site.kaltura.com/rs/984-SDM-859/images/The_State_of_Video_in_Education_2015_a_Kaltura_Report.pdf

Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology, 107(3), 724–739. https://doi.org/10.1037/edu0000013

Learning-theories.org. (n.d.) [Richard Mayer’s cognitive theory of multimedia learning]. [Image]. https://www.learning-theories.org/doku.php?id=learning_theories:cognitive_theory_of_multimedia_learning

McLeod, S. A. (2012). Working memory. Simply Psychology. https://www.simplypsychology.org/working%20memory.html

Malmberg, K. J., Raaijmakers, J. G. W., and Shiffrin, R. M. (2019, January 28). 50 years of research sparked by Atkinson and Shiffrin: 1968. Memory & Cognition, 47, 561-574. https://doi.org/10.3758/s13421-019-00896-7

Mayer, R. (2014). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 43-71). Cambridge University Press. https://doi.org/10.1017/CBO9781139547369.005

Mayer, R. (2014). Incorporating motivation into multimedia learning. Learning and Instruction, 29, 171-173. https://shop.tarjomeplus.com/UploadFileEn/TPLUS_EN_4524.pdf

Miller, G. (1956, March). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2) 81-97. https://pure.mpg.de/rest/items/item_2364276/component/file_2364275/content

MindTools. (n.d.) Cognitive load theory: Helping people learn effectively. [Image]. https://www.mindtools.com/pages/article/cognitive-load-theory.htm

Patwari, R. (2015, April 9). Multimedia principles. [Video]. YouTube. https://www.youtube.com/watch?v=BcWSUnXz8kw

Peshkin, M. (2013, June). Lightboard : a.k.a. learning glass. [Video]. YouTube. https://www.youtube.com/watch?v=N1I4Afti6XE

Schmidt, S. R. (2008). Cognitive psychology of memory. Learning and Memory: A Comprehensive Reference. https://www.sciencedirect.com/topics/neuroscience/dual-coding-theory

Seidel, T., Blomberg, G., & Renkl, A. (2013). Instructional strategies for using video in teacher education. Teaching and Teacher Education, 34, 56-65. https://tinyurl.com/vezdmfd

Slemmons, K., Anyanwu, K., Hames, J., Grabski, D., MLsna, J. Simkins, E., & Cook, P. (2018, May 16). The impact of video length on learning in a middle-level flipped science setting: Implications for diversity inclusion. https://doi.org/10.1007/s10956-018-9736-2

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285 http://csjarchive.cogsci.rpi.edu/1988v12/i02/p0257p0285/MAIN.PDF

Swerdloff, M. (2016) Online learning, multimedia, and emotions. Emotions, Technology, and Learning. https://www.sciencedirect.com/topics/social-sciences/multimedia-learning

Comments are closed.

    Upcoming Workshops

    Picture
    See what is coming up

    Be One of the 1sts to receive notifications about when new posts are uploaded.

    Sign Up

    Interested in exploring digital online tools for teaching? Visit EPDI's sister site at ​https://www.adulted-technology.com/

    Categories

    All
    Assessments
    Attendance And Retention
    Authentic Learning
    Brain
    Collaborative Learning
    Collective Intelligence
    Didactic Teaching
    Diversity
    ESL
    Inquiry Based Learning
    Jigsaw
    Multiculturalism
    Problem Based Learing
    Productive Diversity
    Project Based Learning
    Social Emotional Learning
    Social Mind
    Strategies For Teaching
    Technology
    Theories

    Archives

    April 2021
    January 2021
    December 2020
    April 2020
    February 2020
    January 2020

    RSS Feed

  • Home
  • All Blog Posts
  • Courses
    • Upcoming
    • Create a multimodal Frayer model
  • About
  • Contact