PROFESSIONAL DEVELOPMENT FOR TEACHERS, TUTORS AND TRAINERS
  • Home
  • All Blog Posts
  • Courses
    • Upcoming
    • Create a multimodal Frayer model
  • About
  • Contact
  • Home
  • All Blog Posts
  • Courses
    • Upcoming
    • Create a multimodal Frayer model
  • About
  • Contact
Search

Video as a Learning Aid and the Cognitive Theory of Multimedia Learning

12/21/2020

 
​The use of multimedia in education is increasingly prevalent. This post looks at the cognitive theory of multimedia learning, its sub-theories, and the use of video within the framework of Kalantzis' and Cope's seven principles for new learning and assessment as it relates to technology. 

​Why Video? Video and Multimodality ​

Picture
Video is a form of technology that expresses multimodality. As Cope and Kalantzis point out in Making Sense - Reference, Agency, and Structure in a Grammar of Multimodal Meaning, “meanings are transposable across forms… but in the transposition, the meaning is never quite the same. Each form is partial. Its media have affordances, which offer both opportunities for meaning and constraints. This is why we need multimodality” (Cope and Kalantzis, 2020, p. 33). This is the reason human beings “habitually transpose meanings. Multimodal transposition is in our natures" (2020, p. 33). 

​Cognitive Theory of Multimedia Learning 

Many studies have demonstrated the positive effectiveness of video as an educational tool (Brame, 2015), but the design of the content varies among its producers. Learning as the acquisition of knowledge that results in increased knowledge or changed behavior depends on memory. If memory consists of sensory memory, short-term memory, and long-term memory, their relationship and interaction should be taken into consideration when creating or selecting videos as tools for education – especially the capacity of a learner to capture and hold onto information in his/her memory. Richard Mayer is credited for the cognitive theory of multimedia learning. In addition to using the information processing model, he incorporated Paivio’s dual coding theory, Sweller’s cognitive load theory, and Baddeley’s model of working memory. 

Information Processing Model

​The information processing model of memory treats sensory memory as transient. Part of short-term memory’s function is to decide which data from our senses should be analyzed for processing. The information that is considered too trivial to keep is forgotten; the remainder is encoded and transferred to long-term memory. Short-term memory (working memory) is considered to have limits on how much information it can hold whereas long-term memory in comparison is limitless. George A Miller (1956) theorized that short term memory, can only hold 7 ± 2 bits of information “for about 20 seconds. Further, typical [working memory] can process (i.e., combine, contrast, or manipulate) about 2 to 4 elements of information” (Greer, Crutchfield, and Woods, 2013, p. 42). Atkinson and Shiffrin developed the information processing model in 1968 (Malmberg, Raaijmakers, and Shiffrin, 2019) and Figure 1 illustrates the journey incoming data makes through memory. 
Figure 1

​Information Processing Model
Picture
Information Processing Model. “Adapted from Atkinson, R.C. and Shiffrin, R.M. (1968). 'Human memory: A Proposed System and its Control Processes'. In Spence, K.W. and Spence, J.T. The psychology of learning and motivation, (Volume 2). New York: Academic Press. pp. 89–195. (MindTools, n.d.) https://www.mindtools.com/pages/article/cognitive-load-theory.htm

Dual Coding Theory

​Allan Paivio’s dual coding theory states that the dominant sensory modes are visual and auditory in terms of the way people receive information and that memory uses two separate channels to process the information coming from these senses. Visual information passes through the eyes, sound passes through the ears; the printed word passes through both the visual and auditory channels. Because printed words pass through both channels this has implications for when presenting complex tasks. 
​Concrete language is remembered better than abstract language in a wide variety of tasks” (Paivio et al., 1994, p. 1196). The concreteness effect is most often interpreted within Paivio’s (1971) dual-coding theory, according to which imaginal and verbal processing independently contribute to memory for concrete words, whereas only verbal processing is usually possible for abstract material (Schmidt, 2008). ​
An example of applying dual coding theory is already seen in instruction of low-literacy ESL students. Words that are easily associated with images are taught first in accompaniment with the images that they represent When using the vocabulary in sentences, the students already have the images in their mind associated with the words. and would therefore be able to translate the words better this might be more effective than just having students use a translator to build vocabulary. 
Picture
Image is from Swerdloff, M. (2016).
https://www.sciencedirect.com/topics/social-sciences/multimedia-learning 

Cognitive Load Theory

​Cognitive load theory centers on working memory which is short term. It includes a discussion what interferes with learning as data is processed and analyzed in working memory. It states that because working memory has a limited capacity to hold information (Sweller, 1988), it performs a sort of triage on incoming data transmitted through sensory memory. It categorizes memory into three classes: intrinsic, germane and extraneous. Intrinsic information is that which is perceived to be relevant and vital to achieve the desired learning outcome. Germane information is relevant but not necessary. Extraneous information is irrelevant (though may be interesting) and will not achieve the desired learning outcome. If information is found to be extraneous to the learning objective, that is, it is not intrinsic nor germane to the desired learning outcome, it is disposed of (forgotten) and not sent to long term memory. The more extraneous data working memory is forced to analyze, the longer it takes for working memory to transfer relevant data to long term memory. This may result in learning activities having to be repeated before new knowledge can be considered acquired (Brame, 2015). Chunking information (into familiar bits of knowledge if possible) is recommended. 

​The following video (Bucy, 2009) provides an exercise that quickly demonstrates elements of cognitive load theory. 

Model of Working Memory

​Richard Mayer also credits Alan Baddeley’s model for working memory (WM) as helping him develop his multimedia learning theory. 
Baddeley described three mechanisms within the WM framework that relate to these WM limitations: a central executive (CE) that coordinates the WM processes and two subsystems, a visual/spatial processor, and a phonological loop or auditory processor (Sweller, 2005).The CE has four main tasks: a) focus attention; b) divide attention between more than one task as necessary, c) switch attention back and forth, and d) communicate with long term memory (Baddeley, 2012).The visual/spatial sketchpad stores visual and spatial information and is also responsible for creating and interpreting mental images (Swanson & Saez, 2005).The phonological loop acts as a temporary storage facility for verbal information, including subvocal speech and sounds. These three WM mechanisms interact with incoming information” (Greer, Crutchfield, and Woods, 2013). ​
In 2000, Baddeley added another component to his model called Episodic buffer. Its role is to serve “as a 'backup' store which communicates with both long-term memory and the components of working memory” (MacLeod, 2012). 
In an article issued in January 2020, Baddeley, Hitch and Allen explain the theory’s development since 2000. Mayer’s incorporation of the above theories did not stop those theories from evolving over time as has Mayer’s.
Picture
The image represents the revised multicomponent model of working memory. From “The Episodic Buffer: A New Component of Working Memory?” by A. D. Baddeley, . https://doi.org/10.3758/s13414-019-01837-x

Cognitive Theory of Multimedia Learning

For Mayer, “humans engage in active learning by attending to relevant incoming information, organizing selected information into coherent mental representations, and integrating mental representations with other knowledge” (Mayer, 2014, p. 47) See Table 1 and Figure 2. 
​Table 1. 

The three assumptions of Mayer's cognitive theory of multimedia learning 
Assumption Description
Dual channels Humans possess separate channels for processing visual and auditory information
Limited capacity Humans are limited in the amount of information that can be processed in each channela t one time
Active Processing Humans engage in active learning by attending to relevant incoming information, organizing selected information into coherent mental representations, and integrating mental representations with other knowledge
(Mayer, 2014, p. 47) https://doi.org/10.1017/CBO9781139547369.005

​Figure 2. 

Richard Mayer's Cognitive Theory of Multimedia Learning
Picture
​

Image of Richard Mayer's Cognitive theory of multimedia learning model is from Learning-theories.org. https://www.learning-theories.org/doku.php?id=learning_theories:cognitive_theory_of_multimedia_learning (redrawn to improve sharpness)
The cognitive theory of multimedia learning states that humans intake visual information through one channel, and sound through a second; and that the written word is processed as both. The short-term/working memory processes the information and converts it to mental representations that incorporates what it can from prior knowledge stored in long term memory; however short term memory is limited in terms of the amount it can process, therefore, design of multimedia should take that into account. Humans construct meaningful knowledge when relevant material is selected, organized and integrated with prior learning.
​
In the video below, Rahul Patwari applies multimedia learning principles to a flipped classroom.
Patwari, R. (2015, April 9). Multimedia principles. [Video]. YouTube. https://www.youtube.com/watch?v=BcWSUnXz8kw

​Patwari, besides discussing additional learning principles that come into play with multimedia learning, also discusses the need for chunking information. It is better to have information distributed through five short videos than one long one. So, how long should videos be? Brame (2015) reports that Guo et al examined the engagement levels of students based on video length from four MOOCs. Their results are from 6.9 million video-watching sessions. “They observed that the median engagement time for videos less than six minutes long was close to 100%– that is, students tended to watch the whole video (although there are significant outliers; see the paper for more complete information) (Brame, 2015). Other suggestions have stated that maximum length should be 10-12 minutes.
A study by Slemmons et al found that differences in immediate recall are negligible for long videos but there may be differences in ability to demonstrate understanding over a longer period of time depending on the gender and whether the student has a learning disability. While short-term retention of material did not seem to be influenced by video length, longer-term retention for males and students with learning disabilities was higher following short videos compared to long as assessed on summative assessments. Students self-report that they were more engaged, had enhanced focus, and had a perceived higher retention of content following shorter videos. This study has important implications for student learning, application of content, and the development of critical- thinking skills. This is particularly paramount in an era where content knowledge is just a search engine away. (Slemmons et al, 2018)

Applications for Video in the Classroom: The Seven Principles for New Learning and Assessment

Please note that this article was written earlier in the year before knowing how long the pandemic would last which is the reason the focus is on classroom applications; however, much of this applies to online learning as well.
The theories above have been centered primarily on memory and information processing. A general theory of technology by Kalantzis and Cope (n.d.) suggests how audio/video can be applied in a classroom.
Picture
(Kalantzis and Cope, n.d.) https://cgscholar.com/community/community_profiles/community-16395/community_updates/109824
Ubiquitous learning, active knowledge making, multimodal meaning, recursive feedback, collaborative intelligence, metacognition, and differentiated learning are the seven principles that are core elements of Kalantzis’ and Cope’s reflexive/ergative pedagogy. Reflexive/ergative theory sees the learner as an agent capable of producing knowledge and for this to occur, learning should allow for knowledge that is discoverable and navigational. Learning should also promote knowledge as judgment; knowledge acquired should be representable. Learning should take advantage of social, dialogical minds. Devices in service of learning, therefore, should be “cognitive protheses” (Kalantzis and Cope, 2018).

​Table 2 looks at audio/video media’s capabilities in terms of the seven affordances.
Table 2.

​Using Video in the Classroom and Reflexive/Ergative Pedagogy

The 7 principles of Technology 

Benefit 

Application Example  

ubiquitous learning 

video is accessible through smartphones, iPads, laptops, desktops 

videos can used for flipped classrooms 

active knowledge making 

learners can create their own videos to demonstrate knowledge 

videos can be created from the students’ point of view 

multimodal meaning 

video allows knowledge production using visual, verbal and auditory modes or representation that occur at the same time 

students can include video in Word documents that enhance multimodal expressions 

recursive feedback 

video recordings can be used as substitution to submit regular feedback 

videos can be used by teachers to provide feedback and for scaffolding if the teacher films themselves demonstrating performing a task 

collaborative intelligence 

learners can work together to produce a video 

The production of a video could be an interdisciplinary collaboration 

metacognition 

learners can create video recordings of their reflections-in-action/ reflections-on-action/judgement on learning 

videos can be used as a substitute for the typed word for student self-assessment in the form of self-reflections 

differentiated instruction 

video media can be used as an option in differentiated instruction when it comes to product and process. 

video can be used as part of presenting difficult concepts in multiple ways – lecture plus video and/or as option as to how students can demonstrate their knowledge acquisition. 


​Video has been part of education since televisions could be placed in a cart and rolled into classrooms. With the onset of the Internet and video sharing and hosting web sites, the use of this media has increased exponentially. The rise in online courses has jet-fueled the adoption of video especially in higher education. “Video has become an important part of higher education. It is integrated as part of traditional courses, serves as a cornerstone of many blended courses, and is often the main information delivery mechanism in MOOCs” (Brame, 2015, p. 1)
The majority of these online courses rely on audiovisual instruction supplemented by regular individual or collaborative homework assignments. The format of audiovisual instruction widely varies across courses, but one popular template is a series of roughly 12-min lecture videos showing lecture slides, overlaid with annotation and the instructor’s voice. (Kizilcec, Bailenson, and Gomez, 2015).
​It is true that video have been the domain primarily of instructors in education (Kaltura, 2015), but there is potential in a student-centered environment, during middle school and high school years and in adult education to have students use video as a tool to demonstrate active knowledge making in multiple ways. Videography can be put to innovative use.

Innovation

Please note that this article was written earlier in the year before knowing how long the pandemic would last which is the reason this section is from an in-person teaching viewpoint, but it mention using interactive whiteboards.
When teachers and students think of recording video many think only of a video camera, the video option on their smart phone or laptop. They click the function on and record, but there other video technology exists that can be used for learning. 360-degree cameras can be used to create virtual reality and provide students an immersive experience. Teachers can use neck-mounted or head mounted cameras to film demonstrations from their point-of-view (IPOV – instructor point-of-view). Filming IPOV is easier to do when performing a task, but for instructors who want their students to see them work out a problem or write an example on whiteboard and talk about it as they do that is hard to video record. One could screen record writing on an interactive whiteboard and record the narration separately or an instructor could try using an innovative technology that comes from Northwestern University. It is called a lightboard and is also referred to as the learning glass. This technology allows teachers to write while facing the student. It was created by Michael Peshkin (Fung, 2018).

​In the video below, Michael Peshkin, demonstrates the use of lightboard technology which is open source hardware.
Peshkin, M. (2013, June). Lightboard : a.k.a. learning glass. https://www.youtube.com/watch?v=N1I4Afti6XE
More information about about the open source hardware Lightboard (a.k.a. Learning glass) can be found at https://lightboard.info.

Evidence

Brame (2015) has reviewed multiple studies that have proven the effectiveness of video as an educational tool.  ​Research has also discovered that novice learners benefit from auditory narration rather than reading words when they are also required to process other visual information (Greer, Crutchfield, and Woods, 2013). Carmichael, Reid, and Karpicke (n.d.) found that when an instructor was in the video, watching that instructor perform a task boosted students’ confidence in their ability to accomplish the same task. Research has also found that video-based learning can result in improvements in teaching (Carmichael, Reid, and Karpicke, n.d; Gainsburg, 2009; Seidel, Blomberg, and Renkl, 2013).

Criticisms

​One of the earlier critiques of Mayer’s cognitive theory of multimedia learning is that it did not address motivation. Mayer (2014) reviews some of these studies. In his conclusion he states that “overall, the papers encourage us to consider instructional design features aimed at priming motivation to engage in deep processing during learning, while not overloading the learner’s information processing system” (2014). Another early critique targets long term memory, and its lack of attention compared to short-term memory in Mayer’s theory. In discussing long-term memory one has to bring constructivism. Constructivism talks about the assimilation/accommodation of prior knowledge when acquiring new knowledge. Multimedia learning theories have focused on what happens before information becomes part of long term memory a.k.a. prior knowledge. CTML does not ignore long-term memory. It views the relationship between short-term memory and long-term memory as dialogical.
Endel Tulvig (Harrell, 2020) does focus on long term memory and considers long-term memory to have channels. These channels are more like compartments. One for storing episodic events; another stores semantic information which Tulvig considers general knowledge. There is also separate storage for procedural information. Harrell (2020) mentions the story of Henry Molaison as evidence of this. Henry Molaison suffered a brain injury. Though he remembered how to do certain things, he was not able to remember the context within which he learned to do them.

Conclusion

​The understanding of how a learner processes information is essential for those who use video as a multimodal learning aid. It is important that video presented for learning not overwhelm the brain’s capacity to process and retain information. The research and videos referred to and included in this work discuss decreasing the cognitive load by removing extraneous information and aligning texts near the relevant images discussed. Keeping the videos short can also help. Reducing the cognitive load makes it easier for students to process the relevant information and students find the use of video more engaging.

References

​Brame, C.J. (2015). Effective educational videos. http://cft.vanderbilt.edu/guides-sub-pages/effective-educational-videos/

Bucy, M. (2009, October 18). Cognitive load exercise. [Video]. YouTube. https://www.youtube.com/watch?v=Rc705-WS2l4

Carmichael, M., Reid, A-K., Karpicke, J. (n.d.). Assessing the impact of educational video on student engagement critical thinking and learning: The current state of play. https://us.sagepub.com/sites/default/files/hevideolearning.pdf

Cope, B., Kalantzis, M. (2020). Making sense: reference, agency, and structure in a grammar of multimodal meaning. United Kingdom: Cambridge University Press. https://www.google.com/books/edition/Making_Sense/0YDCDwAAQBAJ?hl=en&gbpv=0

Education at Illinois. (2016 ). E-Learning affordance 3a: Multimodal meaning. [Video]. YouTube. https://www.youtube.com/watch?v=S8fLr9CZg4o

Fung, F. M. (2018, Jun 6). How innovative videography can supercharge education. https://theconversation.com/how-innovative-videography-can-supercharge-education-97676

Gainsburg, J. (2009). Creating effective video to promote student-centered teaching. Teacher Education Quarterly, 36(2), 163-178. https://www.jstor.org/stable/23479258?seq=1

Greer, D. L., Crutchfield, S. A., & Woods, K. L. (2013). Cognitive theory of multimedia learning, instructional design principles, and students with learning disabilities in computer-based and online learning environments. Journal of Education, 193(2), 41–50. https://doi.org/10.1177/002205741319300205

Harell, T. (2020, March 9). What is the modal model of memory? https://www.betterhelp.com/advice/memory/what-is-the-modal-model-of-memory/

Hitch, G.J., Allen, R.J. & Baddeley, A.D. (2020). Attention and binding in visual working memory: Two forms of attention and two kinds of buffer storage. Attention, Perception, & Psychophysics, 82, 280–293. https://doi.org/10.3758/s13414-019-01837-x

Kalantzis, M. & Cope, B. (n.d.). An agenda for new learning and assessment: 7 principles. [Image]. [Course content]. In HRD 472: Learning technologies. University of Illinois Urbana-Champagne. https://cgscholar.com/community/community_profiles/community-16395/community_updates/109824

Kalantzis, M., & Cope, W. (2018). Multiliteracies: Meaning making and literacy learning in the era of digital text. Paper presented at University-Wide Teaching and Learning Symposium organized by Center of Teaching, Learning and Technology. https://ctlt.illinoisstate.edu/downloads/symposium/2018/Kalantzsis-Cope%20Morning.pdf

Kaltura, Inc. (2015). The state of video in education 2015: A Kaltura report. http://site.kaltura.com/rs/984-SDM-859/images/The_State_of_Video_in_Education_2015_a_Kaltura_Report.pdf

Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology, 107(3), 724–739. https://doi.org/10.1037/edu0000013

Learning-theories.org. (n.d.) [Richard Mayer’s cognitive theory of multimedia learning]. [Image]. https://www.learning-theories.org/doku.php?id=learning_theories:cognitive_theory_of_multimedia_learning

McLeod, S. A. (2012). Working memory. Simply Psychology. https://www.simplypsychology.org/working%20memory.html

Malmberg, K. J., Raaijmakers, J. G. W., and Shiffrin, R. M. (2019, January 28). 50 years of research sparked by Atkinson and Shiffrin: 1968. Memory & Cognition, 47, 561-574. https://doi.org/10.3758/s13421-019-00896-7

Mayer, R. (2014). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 43-71). Cambridge University Press. https://doi.org/10.1017/CBO9781139547369.005

Mayer, R. (2014). Incorporating motivation into multimedia learning. Learning and Instruction, 29, 171-173. https://shop.tarjomeplus.com/UploadFileEn/TPLUS_EN_4524.pdf

Miller, G. (1956, March). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2) 81-97. https://pure.mpg.de/rest/items/item_2364276/component/file_2364275/content

MindTools. (n.d.) Cognitive load theory: Helping people learn effectively. [Image]. https://www.mindtools.com/pages/article/cognitive-load-theory.htm

Patwari, R. (2015, April 9). Multimedia principles. [Video]. YouTube. https://www.youtube.com/watch?v=BcWSUnXz8kw

Peshkin, M. (2013, June). Lightboard : a.k.a. learning glass. [Video]. YouTube. https://www.youtube.com/watch?v=N1I4Afti6XE

Schmidt, S. R. (2008). Cognitive psychology of memory. Learning and Memory: A Comprehensive Reference. https://www.sciencedirect.com/topics/neuroscience/dual-coding-theory

Seidel, T., Blomberg, G., & Renkl, A. (2013). Instructional strategies for using video in teacher education. Teaching and Teacher Education, 34, 56-65. https://tinyurl.com/vezdmfd

Slemmons, K., Anyanwu, K., Hames, J., Grabski, D., MLsna, J. Simkins, E., & Cook, P. (2018, May 16). The impact of video length on learning in a middle-level flipped science setting: Implications for diversity inclusion. https://doi.org/10.1007/s10956-018-9736-2

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285 http://csjarchive.cogsci.rpi.edu/1988v12/i02/p0257p0285/MAIN.PDF

Swerdloff, M. (2016) Online learning, multimedia, and emotions. Emotions, Technology, and Learning. https://www.sciencedirect.com/topics/social-sciences/multimedia-learning

Productive Diversity, Multicultural Potlucks and Technology

12/13/2020

 
Picture
​Starting around the time of Thanksgiving through Christmas, teachers in the United States who work with English as a Foreign Language (EFL) learners use the holiday seasons to give their students an opportunity to share their culture through food. 
​These social events at schools and in classrooms give students an opportunity to take on the instructor role, as they describe their dish, how it is made and any historical significance. The EFL learners practice English while sharing a significant part of who they are.
​A question is how to bring that experience and its benefits to an online community? What technology
can be used? The experience is not just bringing the food but it encompasses tasting the food and having
discussions around it.
​Using Bloom’s Taxonomy, here is an outline of how this could work in an online community.
​1. Remember – Students from different countries or regions would be paired. They would recall a dish from
their home country, write a recipe for it in their home language and exchange it with their partner using an
online drive.
2. Understand – Student would translate the recipe they received – ingredients, measurements, and
instructions - into English. The sender of the recipe would provide feedback as to whether they thought
the translation was accurate.
3. Apply – Each student would prepare two dishes. One would be the recipe received from their partner; the
other would be from the recipe they gave their partner.
4. Analyze – The day of the potluck students would exchange photographs of their dish showing the outside
and inside and would interact using video chat software that had a/v recording capabilities. They would
hold up the same dish and in English compare, contrast, identify and categorize the tastes, smells and
explain any substitutions they may have made in English.
5. Evaluate – They would recommend to each other changes, if needed.
6. Create – The students would brainstorm and plan a fusion dish that had elements of recipes from each of
their country; and prepare the new dish for when the whole class met.
7. The pairs would meet online with the whole class, provide background on the dishes they chose to share,
and taste their fusion dish and discuss the results.
Picture
To summarize, in order to make a multicultural potluck work online, the digital technology needed would be file-sharing and video
conferencing.

The Value of Self-Assessments

12/6/2020

 
Picture
Self-assessment is the process of ongoing reflection, self-judgment, and self-monitoring to summarize one’s strengths and clarify areas for improvement. This includes learning from one’s experiences; judging one’s personal, educational, and professional performance; examining personal characteristics based on evidence, external standards, and explicit criteria for the purpose of future quality and performance improvements. Research has shown that self-assessment can improve learner's communication skills, and can result in behavior change (Nonyel, 2015).
Nonyel’s definition as quoted above focuses on the process and stops at incorporating the results of the process into the definition. McMillan and Hearn’s definition is similar except that they add “identification and implementation of
instructional correctives as needed” (2008, 41). They also state that the learners “provide feedback to themselves based on well-understood standards and criteria.” It’s more than just a reflection based on observation and experience; it is formative and based on external criteria. In the classroom that criteria is offered often in the form a rubric. The attached rubric is an example of a self-assessment rubric from a Differentiated Instruction course.  The students can use it to self-assess the lesson plans they created. The criteria are graduated and task-specific.
differentiated_instruction_lesson_plan_rubric_by_katriel_c_jean-baptiste.pdf
File Size: 123 kb
File Type: pdf
Download File

The inclusion of a rubric as part of a definition of self-assessment makes it more teacher-centric – assuming that the instructor is the person who created the rubric and not the learner. It can be argued that this is necessary in formal, organized learning situations; however, there are times when a rubric is not as important or should not be part of the equation. For instance, a learner may register for a webinar, or attend a conference with personal learning objectives that are a subset of the instructor, presenter, or facilitator.
Picture
It is the standard for webinars and conference workshops to not have rubrics. Also, self-assessment may begin before the learner is in the class. The enrollment or registration may be the beginning of the “implementation of instructional correctives” (McMillan and Hearn, 2008, 41)

Background and Interest

My interest in self-assessments comes from some experiences I have had as a teacher. At the beginning of a lesson or new topic I will usually ask students what is their knowledge of or experience in that topic. A few students raise their hand to say that they have significant or some knowledge or experience; however, ensuing informal dynamic assessments and activities show that that at least one of the students who responded had rated their skills or knowledge higher than s/he should have. I have also experienced the opposite, where a student rated themselves lower than I would have but that has happened significantly less often. Self-assessment is a component of many learning theories. I decided to look further into this topic to see whether my anecdotal observations have also been observed in research studies and if yes, how useful are student self-assessments.

Theory of Self-Assessment

Mary Kalantzis and Bill Cope (n.d.) list the following principles for general assessment in the classroom.
1. Assessments over a learning unit should consist of formative and summative assessments
2. Assessments should test the “full range of knowledge process required in the Learning Element;’’ the set should
cover the whole learning unit or a “special assessment task” such as a concluding project.
3. Assessments should be recursive; ubiquitous.
4. Assessments should involve multiple people: learner, peer, parents (if learners are children), subject matter
experts; invited critical friends
5. Assessments should determine the quality of knowledge and performance in the domains of “experiential,
conceptual, analytical and applied.”
6. Measuring imagination, metacognition, problem solving, teamwork and multimodal expression should be standard.
7. Measurement of teamwork includes the ability to collaboratively construct knowledge and make productive social
connections.
8. Peer reviews include open, one-way blind, two-way blind, and moderated
9. Assessments should measure learners’ ability to extract knowledge from collaborations and other resources of
knowledge such as experience, research articles, observations and apply that knowledge.
10. Assessments should be able to justify quantitative ratings with qualitative judgments.
11. Portfolios are part of assessments; they include objective evidence of what has been learned, ratings, and
commentaries and “not just what you can infer they have learnt in an end-of-program test.
Picture
Using Nonyel’s (2015) definition of self-assessment as “reflection, self-judgment, and self-monitoring to summarize one’s strengths and clarify areas for improvement”, then self-assessment comes into play in item 4 (ongoing reflection); item 6 (metacognition, problem-solving); items 10 and 11 (reflection). This leads to a theory of self-assessment as a recursive process whose goal is to solve a problem or actualize a concept; and the process must create metacognition. If this process is recursive, then opportunities for self-assessment must be given throughout a course of study which suggests having multiple rubrics instead of one provided for final outcomes which is summative and not formative. Does this matter if self-assessments are not accurate and vary from teacher assessments?

​Evidence of Problems with Self-Assessment

Jean-Baptiste, K. C., (2020). Students’ self-assessment compared to teachers.[Video].
In a review of research evidence on student self-assessment, John A. Ross (2006) asks four questions (p. 1):
1. Is self-assessment a reliable assessment technique?
2. Does self-assessment provide valid evidence about student performance?
3. Does self-assessment improve student performance?
4. Is self-assessment a useful student assessment technique?
Ross concluded for the first question that “adequate consistency involved students who had been trained in how to
evaluate their work” but the consistency waned over longer periods of time. This decrease in consistency was acute with young children; and it varied among subject matter (2006). Using as a definition for validity to mean “agreement with teacher judgments” he found the results mixed in part due to “the criteria used by teachers and students were frequently not defined; there were few replications involving comparable groups of students,” and “self-grading was not defined” (2006, p. 3). Ross further goes on to discuss validity in terms of evaluating evidence that self-assessment does improve student performance. In answering the third question he discusses validity in terms of consequences.
Consequential validity is the argument that the worth of a test is determined by its consequences for students and others. For example, a valid assessment is one that contributes to student learning—if the assessment has a negative effect on student learning, the test is invalid (Moss, 1998)” (Ross, 2006, p. 4).
The studies Ross reviewed found that students who participated in self-assessment did have positive achievement outcomes. Because of this, he concludes that self-assessment is useful (2006, p. 7), especially if the students are cocreators of the rubric for self-assessment and are trained on how to apply it.
Heidi L. Andrade (2019) reviewed research primarily conducted between 2013 and 2018 on student self-assessment. Like what Ross (2006) found, the results of the studies that Andrade reviewed showed mixed results for consistency (in terms of comparing scores to teachers) but a “positive association between self-assessment and learning”
(2019, p.8). She also found that the type of external criteria used made a difference. Students who were given performance-based criteria for self-assessment as opposed to competence-based rubrics performed better
​(2019, p. 9).
Besides achievement, improvement of learner's communication skills (Nonyel, 2015), do other benefits exist?
The following video on self-assessment shows how some students view self-assessment as a benefit because it provides clarity. Also demonstrated is a way to create self-assessment for students in this elementary school. Studies show that students do better when they are co-creators of the rubric. (Ross, 2006).
JFF. (2013, August 22). Self-assessment: Reflections from students and teachers. [Video]. Youtube.
https://www.youtube.com/watch?v=CkFWbC91PXQ
Some data suggest that students prefer self-assessment to assessment by the teacher alone. The reasons given by grade 5-11 students why they preferred self-assessment suggest additional benefits of self-assessment: 1) students said that with self-assessment they had a better understanding of what they were supposed to do because they were involved in setting the criteria for the assessment; 2) students argued the self-assessment was fairer because it enabled them to include important performance dimensions, such as effort, that would not usually be included in their grade; 3) self-assessment enabled them to communicate information about their performance (e.g., their goals and reasoning) that was not otherwise available to their teacher; 4) self-assessment
gave them information they could use to improve their work (J. Ross et al., 1998). These changes in perception in the value of assessment through greater involvement in the process might reduce the trend reported by Paris, Lawton, Turner and Roth (1991) in which students become increasingly cynical about the validity and value of assessment as they move through the school system. (Ross, 2006, p. 7)
Ross (2006) also looked at how teacher’s viewed student self-assessment. Responses included that students were more engaged, especially if they were involved in the process; also students learned more.
Not all studies showed that students or teachers found benefits. Some students thought it was “boring” or felt that it was the instructor’s responsibility to assess students and students should not be involved. (Ross, 2006). The following Table 3 by Chris Andrews lists teacher response to interview questions regarding student self-assessments (SSA).
Reported Reasons for Using and Not Using SSA Reasons for using SSA (16 different reasons) Student Benefits (62% of reasons for using SSA) SSA helps students learn how to think and problem-solve (Alan, Amy) SSA helps students see the connection between work, practice, and performance (Alan) SSA helps to see students’ understanding (Alan, Amy) SSA helps students to grow as learners (Jason, Ryan) SSA is an important life-skill (Amy, Jason) SSA helps students monitor and adjust their behavior (Alan, Greg) SSA gives students more ownership over their learning (Amy, Greg, Jason, Ryan) SSA helps provide a safe environment for students to express their thoughts (Greg) SSA builds resiliency in students (Greg) SSA motivates students (Amy, Greg) Teacher Benefits (19% of reasons for using SSA) SSA helps the teacher give more specific feedback (Alan, Amy) SSA helps the teacher get a better picture of student learning (Amy) SSA saves a teacher time in assessment (Amy) External Influences (19% of reasons for using SSA) SSA is part of the state core drama standards (Amy) Teacher was trained with SSA (Amy) SSA aligns with principal’s expectations (Amy) Reasons for not using SSA (8 different reasons) Time (50% of reasons for not using SSA) SSA might take too much class time (Alan, Kathy, Sean) SSA might take too much time to learn how to implement (Alan, Sean) Not having a complete understanding of SSA (Alan, Sean) Lacking good models of SSA (Sean, Greg) Student Capability (37.5% of reasons for not using SSA) Students do not know how to assess themselves accurately (Kathy, Sean) Students sometimes grade themselves too harshly (Kathy) Some students have lower expectations (Kathy) Teachers’ Responsibilities (12.5% of reasons for not using SSA) The teacher is responsible for assessment in the classroom, not the students (Kathy)
Student self-assessment: Teachers’ definitions, reasons, and beliefs. [Thesis]. Indiana University Bloomington. https://www.researchgate.net/publication/318927776_Student_selfassessment_ Teachers%27_definitions_reasons_and_beliefs

Application and Innovation

Multiple tools exist that provide an opportunity for students to practice self-assessment: external criteria such as
performance- and competence-based rubrics; LMS’s such as Common Ground Scholar created by Mary Kalantzis and
William Cope which has a built-in review component. The KWL graphic organizer asks learners to assess what they know, what they want to learn, and what they have learned about a subject. Other opportunities are online, quiz makers that can be designed to have students think about how they feel about what they are learning.. One of them is Bookwidgets.com. A link from one of their blog posts illustrates how it might be done (MyWorksheetAssessment (bookwidgets.com).
Another self-assessment tool comes from Brainscape.com. Brainscape is a company that has an innovative way to use digital flashcards. The software uses an algorithm that recycles flash cards based on a learner’s confidence about the answer. The assessment is based on the confidence that learner feels about knowing the answer. Terms for which the student expressed low confidence are repeated more frequently than others.
Brainscape is a synchronous web and mobile flashcard program designed to improve the retention of declarative knowledge. It is different from other spaced-repetition flashcard programs in that its pattern for re-assessment is based not on a random algorithm nor on the user’s past history of correctness, but rather on the user’s own judgment of confidence in each piece of information – a process that Brainscape calls ConfidenceBased Repetition (CBR). Cohen (n.d.)
Below are screenshots of user interfaces from Cohen's (n.d.) article, Brainscape’s Confidence-based Repetition
Methodology. The student receives the prompt, then after tapping the virtual button are asked to express the judgment of learning - how confident they felt that they knew the answer. It keeps the history of responses and can display the history by single or across multiple subjects for each learner.
Picture
Picture
In a blog post about the benefits of self-assessment, Cohen (2017) addresses three criticisms about self-assessments.
1. How do you know the learner will accurately assess his/her knowledge?
2. How do you know learners won’t avoid thinking about how they feel about the answer and just reveal it?
3. What happens if a learner reports with high confidence their feeling about the answer and is wrong?
In terms of accuracy he says that since this is a self-study application there is no motivation to cheat. His answer to the second question is not strong. He simply states that flash cards are a good tool for when learners are pressed for time and that it's better than multiple-choice and matching questions “from a cognitive standpoint since they merely test recognition rather than engaging active recall” (Cohen, 2017). His response to the third question is that corrections to wrong answers for which learners felt very confident can “yield better retention benefits than if the confidence was never misjudged in the first place" (2017). For his response he refers to Butterfield and Metcalfe's 2006 article, "The Correction of Errors Committed with High Confidence" in Metacognition and Learning.

Conclusion

I wanted to explore in this paper whether student-self assessments are consistent with teacher assessments, and if it is not, what is its value? Studies have shown that inconsistency does exists between teacher and student self-assessments; however, self-assessments are positively correlated with achievement. Results of studies suggest that achievement can be further increased when students have an external criteria to use for self-assessment; especially if they co-created the external criteria, or received training on how to interpret and apply the external criteria. The value of self-assessment, then is not in its reliability or accuracy but in the resulting rise in achievement when it is used; and perhaps it should only be used as formative and not made part of any summative assessment that is tied to grades.

References

Andrade, H. L. (2019) A critical review of research on student self-assessment. Frontiers in Education. 84. https://doi.org/10.3389/feduc.2019.00087

Andrews, C. (2016, June). Student self-assessment: Teachers’ definitions, reasons, and beliefs. [Thesis]. Indiana University Bloomington. https://www.researchgate.net/publication/318927776_Student_selfassessment_
Teachers%27_definitions_reasons_and_beliefs

BookWidgets (n.d.). [Screenshot of an online student self-assessment]. From https://www.bookwidgets.com/play/SCKZLD

Cohen, A. S. (n.d.) Brainscape’s "Confidence-Based Repetition" methodology. https://www.brainscape.com/images/cms/research/Confidence-Based_Repetition.pdf

Cohen, A. S. (2017, April 18). The benefits of self-assessment. https://www.brainscape.com/blog/2010/04/benefits-of-selfassessment/

Jean-Baptiste, K. C. (2019). Differentiated Lesson Plan Rubric.

Jean-Baptiste, K. C., (2020). Students’ self-assessment compared to teachers.’ [Video].

JFF. (2013, August 22). Self-assessment: Reflections from students and teachers. [Video]. Youtube.
https://www.youtube.com/watch?v=CkFWbC91PXQ

Kalantis, M. & Cope, B. (n.d.) Assessment. https://newlearningonline.com/learning-by-design/assessment

​McMillan, J. H., & Hearn, J. (2008, Fall). Student self-assessment: The key to stronger student motivation and higherachievement. Educational Horizons, 87(1), 40-49. https://files.eric.ed.gov/fulltext/EJ815370.pdf

Nonyel, N. P. (2015, June 19). Self-assessment is essential to lifelong learning. Educational theory and practice.
https://edtheory.blogspot.com/2015/06/self-assessment-is-essential-to.html

Ross, J. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research & Evaluation. 11(10). https://pareonline.net/pdf/v11n10.pdf

Webb, S. (n.d.). [Training : Woman looking in the mirror]. [Image with added text]. https://pixabay.com/photos/trainingmuscles-arms-blonde-828726/

    Upcoming Workshops

    Picture
    See what is coming up

    Be One of the 1sts to receive notifications about when new posts are uploaded.

    Sign Up

    Interested in exploring digital online tools for teaching? Visit EPDI's sister site at ​https://www.adulted-technology.com/

    Categories

    All
    Assessments
    Attendance And Retention
    Authentic Learning
    Brain
    Collaborative Learning
    Collective Intelligence
    Didactic Teaching
    Diversity
    ESL
    Inquiry Based Learning
    Jigsaw
    Multiculturalism
    Problem Based Learing
    Productive Diversity
    Project Based Learning
    Social Emotional Learning
    Social Mind
    Strategies For Teaching
    Technology
    Theories

    Archives

    April 2021
    January 2021
    December 2020
    April 2020
    February 2020
    January 2020

    RSS Feed

  • Home
  • All Blog Posts
  • Courses
    • Upcoming
    • Create a multimodal Frayer model
  • About
  • Contact