The Three Things I Believe About Assessment: Revised

The first act in CEP813 was to write about three things I believe about assessment. This is a great approach to set up a student to have an anchor point to reflect on how their concepts, assumptions, and beliefs have been transformed by learning new concepts. This is certainly the case with my experience in CEP813.

My first stated belief at the beginning of the course was that multiple-choice questions are virtually worthless. Of my three stated beliefs, this is the viewpoint that has shifted the most. As I wended my way through the course, Toward the beginning of the course my notions regarding multiple-choice questions were put to the test. In Module 1 we considered the text, Knowing What Students Know, published by the National Academy of Sciences. The authors of Knowing What Students Know assert that assessment should rest on three pillars, one of which is “tasks or situations that allow one to observe students’ performance” (National Science Foundation, 2001, p.23). This led me to wonder, can a multiple-choice question satisfy the requirement of a teacher observing a particular task to elicit a particular performance task that provides the teacher with an essential understanding of a student’s proficiency? The conclusion I arrived at was that multiple-choice questions could play a constructive role in doing so. How so? Certainly multiple-choice questions on their own are extraordinarily limited as tools of assessment, however, they can serve as a doorway toward deeper thinking and reflection.

This insight led me to use multiple-choice questions as a quick check of a students grasp of a particular skill. For example, having students read a passage of text and then taking them to use context clues to determine the meaning of a particular word turned into a doorway where, if the student demonstrates a lack of proficiency in the skill, they could write a metacognitive reflection as to why they had the misconception and what they did to correct it.

My second stated belief was that assessment should be integrated as a part of how students learn. While what I learned in CEP813 affirmed this position the course changed the core reason’s I believed so. Chief among these was our consideration of the importance of feedback. In my reading of Hattie and Timperley, I learned that “Too often, assessments are used to provide snapshots of learning rather than providing information that can be used by students or their teachers” (Hattie. Timperley, 2007, p. 104). This made me rethink what it meant for an assessment to be an integrated part of assessment As a consequence I explored ways I could provide students immediate feedback on my assessment designs and I discovered that technologies such as Google Forms afforded me the ability to guide students through a digital assessment embedded with automated feedback contingent upon their responses to various test items.

My third belief was that assessment should be designed to provide effective feedback to students. As you can see, I now see effective feedback and assessment as a learning activity as an integrated aspect of effective assessment. Helping students gain mastery of the language and skills specific to the discipline being assessed, in my case history, is essential to student proficiency. I thought I understood James Paul Gee’s concept of the Semiotic Domain but it was not until I created my game based Assessment that I truly appreciated what he was proposing (Gee, 2001). Formulating the game-based assessment engaged me in thinking more concretely about the role procedural language in assessment and how such an assessment can not only measure the degree to which a student has gained command of the semiotic domain of a particular disciple but also can serve as an assessment for learning at the same time.

References

James Paul Gee, Schallert, D. L., Hoffman, J. V., Maloch, B. E., Worthy, J. E., & Fairbanks, C. M. (n.d.). 51St Yearbook of the National Reading Conference (San Antonio, Texas, December 5-8, 2001)., pp. 23-32 Publication year: 2001

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. Retrieved from http://www.jstor.org.proxy2.cl.msu.edu/stable/4624888

National Research Council. 2001. Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: The National Academies Press. https://doi.org/10.17226/10019.

Formative Assessment Design 3.0

For my pre-assessment instruction I am targeting three specific goals for my students:

1. CONTEXT CLUES: Determine the meaning of a word or term in a historical text by using context clues.
2. THE MAIN IDEA: Identify the main idea in a historical text
3. SUMMARIZE: Express their own understanding of the main idea of a historical text through writing a summary of the main idea in their own words

This will bring focus to my instruction because these goals of the FAD will drive the objectives of my lesson planning. Embedded in both my instruction and FAD 3.0 is an emphasis on promoting peer to peer and student to teacher dialogue as an essential part of how students receive and process feedback (Nicole, McFarland-Dick, 2006, p.209).  

CONTEXT CLUES

• For pre-assessment instruction, I will model using context clues to determine the meaning of a specific word. Before instruction, I will select a specific word that is not common in high school vocabulary. During instruction, I will identify the word I wish to understand the meaning of and then do a think aloud as I read the selected text. A think-aloud is writing down and verbally explaining my thinking process as I read. As I read, I will describe how the surrounding sentences and text lead me to the definition of a particular word.
• Following my modeling activity, students will work in pairs to perform the same context clue task with a different word provided to them. Once the pairs have completed the task they will compare their results to the other pair of students at their tables identifying what clues they had in common and what clues they did not.
• In my FAD 3.0, students will perform the same task except in the FAD 3.0 students will select a statement that best matches what they believe the definition of the word is based on their context clues analysis. Whether the students select the correct answer or not, they must explain the context clues they identified and used to determine the correct answer or the context clues they used in a second attempt at determining the correct answer.
• With the assessment complete there are two specific ways I will use the resulting data. First, students will compare their results with one of their Compass Partners. Compass partners are a set of peers that each student selects as their preferred fellow students for sharing and peer-reviewing each other’s work for peer feedback. 
• Because my FAD 3.0 is a Google Form, I have access to instant results of the multiple-choice items which will allow me to identify students I wish to work with during the peer review process. As students work with one another in their peer review and feedback I will circulate among the students, focusing on students whose assessment scores suggest they need supplemental instruction and support. As I work with different Compass Partner pairings I will be listening carefully to the peer feedback. Depending on the content of the peer feedback I may ask further questions to push student thinking or offer further direct instruction to target areas of demonstrated need.
• Secondly, I will analyze the results of the FAD 3.0 to determine if certain questions sets were problematic for most students across class sections and groups of students. If I determine that there were questions that had a high percentage of error I will review both the structure of the question itself to identify whether the error percentages were due to a poorly written question or if further whole class or group supplementary instruction is required.

THE MAIN IDEA

• For pre-assessment instruction of identifying the main idea, I will use the think-aloud approach again. In this case, I will obviously think aloud my reading and explain the critical reading strategies I am using in identifying the main idea of a text. The main strategy will be to prioritize the importance of sentences within the text from the most essential to the least. In my modeling, I will select what I believe are the three most important sentences or passages of text and explain why I believe so.The most essential sentence or passage of text is the one that best expresses the main idea of the text entire text.
• Following my modeling activity, students will work in pairs to perform the same task of identifying and prioritizing the three sentences or passages of text that are most essential to the main idea of the paragraph. Of course, students will be provided a different text than the one I used to model the task. Once the pairs have completed the task they will compare their results to the other pair of students at their tables identifying what three sentences or passages of text are most essential to the main idea as well as prioritizing from 1-3 sentences or passages of text most essential to the main idea.
• In my FAD 3.0, students will perform the same task except in the FAD 3.0 students will select a sentence that best represents what they believe the main idea of the text is.
• With the assessment complete there are two specific ways I will use the resulting data. First, students will compare their results with one of their Compass Partners. Compass partners will compare their results and examine and explain to one another why they think they selected the correct or incorrect answer.I will use the results of the FAD 3.0  to identify individual or small groups of students who I wish to provide follow-up instruction. As the students compare and analyze the reasons behind their choices analyze I will offer questions to push student thinking or provide further direct instruction students I have identified as requiring further instruction and support.
• Secondly, I will analyze the results of the FAD 3.0 to determine if the question was problematic for most students across class sections and groups of students. If I determine that the question had a high percentage of error I will review both the structure of the question itself to identify whether the error percentages were due to a poorly written question or if further whole class or group supplementary instruction is required. 

SUMMARIZE

• For pre-assessment instruction of summarizing the main idea I will again use the think aloud approach.Using the same text I used to identify and prioritize the three most essential sentences or passages of text to the main idea, I will model by thinking aloud how to identify words essential to understanding the text from the top three sentences. Following that I will model how to reformulate the language into my own words as a model for the students how they can best summarize the main idea in 3-5 sentences.
• Following my modeling activity, students will work individually— unless paired with a partner in certain exceptional cases— to perform the same task of identifying key words and terms and writing a summary of the main idea in their own words. Of course, students will be provided a different text than the one I used to model the task. Additionally students will be provided a “Summarizing the Main Idea” rubric. Once students have completed the task they will compare their results to their shoulder partner at their table for peer review and peer feedback. Peers will use the “Summarizing the Main Idea” rubric in order to provide feedback. I will circulate among students to ensure students remain on task and support students as I see the need.
• In my FAD 3.0, students will perform the same task of summarizing the main idea. With the assessment complete there are two specific ways I will use the resulting data. First, students will compare their summary of the main idea with one of their Compass Partners. Additionally, compass partners will provide feed-back to their peers using the  “Summarizing the Main Idea” rubric. As the students compare and analyze summaries, I will offer questions to push student thinking or provide further direct instruction students I have identified as requiring further instruction and support.
• Students then will have the opportunity to revise their summaries by incorporating the feed-back from peers and myself based on the  “Summarizing the Main Idea” rubric. 
• I will score all summaries with the  “Summarizing the Main Idea” rubric. If I find that a large percentage of students received a low rubric score on one specific criteria I will engage students in further instruction and reflection on that particular criteria and create space for meaningful peer to peer and student to teacher dialogue based on student reflections on their understanding and mastery of that particular rubric criteria.

Please explore or even take my quiz here

References

Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/0307507060057209

Embedded in a Medieval French Village During the Black Plague

 

In CEP813 we have had the opportunity to be introduced to a simple game-based technology called Twine which can be found at twinery.com. With this platform, you can provide students with a game-like activity in which they can make choices that lead them through various activities that address their instructional needs for mastery of knowledge and/or skills. I had the pleasure of playing one of my peer’s games who happens to be a history teacher as well. For their game, the player is essentially embedded in a small village in Medieval France as an elder in the community. The village faces the specter of the Black Death as it sweeps across the southern regions of France. As a village elder, the player is faced with various scenarios. For example, whether to read a letter from the King or an itinerant priest. Then the player must decide what to do based on what they read in the letters. Depending on those choices determines the fate of the village.

Because I am too, a history teacher I expected to perform well in the game. I expected that as I played the game the outcomes would clearly demonstrate to me how I performed. As I played the game, I understood where the teacher wished to take players. Because much of the Medieval belief system was based on superstition, many of the choices students faced were grounded in those superstitions. The point of the game became clear toward the end of the game. Ignoring the King’s command to arrest and kill the itinerant priest led to the village’s survival whereas choosing to arrest and hang the priest led to the demise of the village within two weeks.

While the game was fairly easy for me to understand it holds promise in captivating students as they attempt to navigate a very unfamiliar landscape. I predict many students will make choices that lead to the demise of the village and this will be an engaging surprise for them.

As an assessment for learning, this particular game accomplishes that goal. That is the cool thing about making a twine- students have the opportunity to go back, reflect, and rethink their thinking process if they went down an undesirable path. The only suggestion I would make regarding this game is perhaps students could be alerted to the fact that the fate of the village rests in their own hands. That way the students understand that the survival of the village is the signal of their success in playing the game.

The Game: 2.0

The game I have designed in Twine seeks to reinforce the internal and external grammar of historians. It will reinforce internal grammar by tasking students to engage in specific historical thinking tasks to successfully navigate their way through the game. This means they must pay careful attention

One of the skills that students focus on mastering in my classroom is historical-critical reading skills such as corroboration of evidence and sources, analyzing the historical context of sources, vetting the credibility of a secondary source, and determining the bias and frame of reference from a primary or secondary source.

The twine game will be set up to help students to either corroborate multiple documents and/or determine if they are an appropriate source for responding to a historical problem.

The game will seek to reinforce the internal and external grammar of historians. It will reinforce internal grammar by tasking students to engage in specific historical thinking tasks to successfully navigate their way through the game. For example, for internal grammar, exposing students to content that historians would deem appropriate for the historical practice of interpreting the past, namely, primary source documents.

When students navigate through the game they must successfully determine which side of a historical argument a particular primary source would support. Students, then, must engage with internal grammar, eg. primary sources, historians deal within their work. This is the essence of the external grammar of historians- that they allow all the evidence lead them to their interpretation and conclusion by means of corroboration of all the available primary source evidence

• Students will be presented with the following historical problem:

Should President Harry Truman have dropped the atomic bombs on Japan or was this the wrong decision?

• Students will answer the question before weaving their way through a series of documents related to the decision to drop the atomic bombs on Japan.
• For each document, students will decide whether the United States should or should not have dropped the bomb.
• Based on their choice, students will then consider a document that is strong evidence against their previous choice. An example of this is Harry Truman’s explanation for dropping the bomb. This source most obviously supports the position that the dropping of the atomic bomb was necessary. If a student were to make the choice that Truman’s rationale for dropping the bomb was necessary then they would move on to Admiral William E. Leahy’s memoir in which he states that the bomb did not contribute any advantage to defeating Japan. This document supports the argument that the U.S. should not have dropped the atomic bomb. If the student, however, chose to use the document to support the dropping of the bomb they would be exposed to another document that also is evidence against dropping the bomb thus providing the student an opportunity to evaluate a document that does not support Truman’s decision
• At the end of the Twine, students will be tasked to take a final position to the question and then collect their evidence from the Twine activity that corroborates with one another and supports their position. For example, students will be presented with all the primary source readings in the twine and then take a position as to whether the bomb should have drooped or not and select the primary sources that best supports their argument. This final task in the twine is what would be used to assess student’s command of evaluating primary source documents, analyze for bias, and corroborate supporting primary source evidence.

So how will the assessment meet the expectations of Assessment Design Checklist?

---

1. QUESTION: Does my assessment task students to express how they have made meaning of the knowledge and concepts?

EVIDENCE: My assessment tasks students to make practical application of knowledge, theories, and concepts to novel problems

This assessment provides students the opportunity to make meaning of the content (internal grammar) through using historical thinking skills to make their own independent position regarding the historical problem presented in the assessment. 

---

2. QUESTION: Does my assessment task students to apply thinking skills in a manner where they must transfer their knowledge of concepts to new and/or unfamiliar contexts?

EVIDENCE: My assessment tasks students to apply their understanding of learned concepts by synthesizing, analyzing, and evaluating problems embedded in new and/or novel contexts.

While this is, obviously, an old historical problem that historians have grappled with for decades it is a novel historical problem for the vast majority of students. To make meaning of this content they must apply, or transfer, the historical thinking skills they have learned in a new and novel context.

---

3. QUESTION: Does my assessment aide in reducing the discrepancy between current and desired understanding?

EVIDENCE: Feedback provided to the student from the assessment helps the student improve performance in the targeted task in a measurable manner.

This assessment accomplishes this goal by allowing students to work independently by applying historical thinking skills to content. When they evaluate a primary source incorrectly, they are directed to another document which possesses the same bias thus providing the student with the opportunity to identify their own discrepancy. 

---

4. QUESTION: Does my assessment offer students a space to reflect on their learning of targeted skills and content?

EVIDENCE: Students write reflections identifying how they corrected misconceptions. Students will specifically write about their understanding and mastery of targeted content and skills.

For the final task, students take their own position on whether the US should have dropped the atomic bombs on Japan. The prompt requires them to select two of the sources they believe supports their position. They will have to explain how each of the sources they select supports their claim. Once they submit their writing I will review their work and determine whether the sources they selected to support their position. Once I have evaluated students’ responses I can determine which students have gained mastery and which students require further opportunities to master the skill. What I choose to do instructional will depend on how many students demonstrate mastery versus how many students require further practice.

Based on my previous experience with similar exercises I predict that enough students will need further instruction and time to practice the skill. In that event, I will assign students who have gained mastery to formulate an argument opposite of the one the made for their initial claim.

Take my Twine for a spin

 

The Game: 1.0

History as Semiotic Domain

After briefly considering James Paul Gee’s concept of semiotic domains I have pondered over what implications this has in my world as a history teacher. What types of modalities communicate distinctive types of meanings within the discipline of history? To parse this down even further, Gee offers two other related concepts- internal grammar and external grammar. 

When it comes to internal grammar, what are “the principles and patterns” that can be “recognized as what is and what is not acceptable or typical content” in history? (Gee, 2003, p. 30) In the case of history, acceptable content includes primary sources, secondary sources, and tertiary sources. Credible evidence used to support historical arguments, Cause and effect relationships, determining the bias of any work from historians or persons for history, and corroboration of sources are just a few of the “principles and patterns” that define history as a semiotic domain.

When it comes to external grammar, “what is and what is not an acceptable or typical social practice and identity” in the field of history? (Gee, 2003, p. 30) One thing that is not acceptable is using historical evidence out of context to support one’s own point of view.

Brainstorming a Game for History

After reading Bogost, and then exploring Twine, my initial brainstorming regarding twine as a form of formative assessment is beginning to take form. I think history is an excellent fit for this technology. History is so often defined by the cause and effect of one historical development to another. My initial thinking is that I could create a twine that considers alternate histories based on some of the possible choices certain historical actors could have possibly made and what the potential outcomes could have been. Another twine might walk students through a series of primary and secondary sources and make choices about what evidence they would use to support an argument. One of the questions from my  Assessment Design Checklist is: does my assessment task students to apply thinking skills in a manner where they must transfer their knowledge of concepts to new and/or unfamiliar contexts? I believe constructing a twine for this particular instruction tasks students with having to transfer skills and knowledge to new and unfamiliar context.

The Game

One of the skills that students focus on mastering in my classroom is historical critical reading skills such as corroboration of evidence and sources, analyzing the historical context of sources, vetting the credibility of a secondary source, and determining the bias and frame of reference from a primary or secondary source.

The twine game will be set up to help students to either corroborate multiple documents and/or determine if they are an appropriate source for responding to a historical problem.

• Students will be presented with the following historical problem:

Should President Harry Truman have dropped the atomic bombs on Japan or was this the wrong decision?

• Students will answer the question before weaving their way through a series of documents related to the decision to drop the atomic bombs on Japan.
• For each document, students will decide whether the United States should or should not have dropped the bomb.
• Based on their choice, students will then consider a document that is strong evidence against their previous choice.
• At the end of the Twine, students will be tasked to take a final position to the question and then collect their evidence from the Twine activity that corroborates with one another and supports their position.

Assessing Google Forms as a Tool of Assessment

For my CMS assessment, I have selected Google Forms. Though Google Forms is not unfamiliar to me I am a recent adopter in using Google Forms for assessment. The hustle and bustle of the school year left little to no time to explore the features of Google Forms in more depth and therefore it is very fortuitous that I have had the opportunity to do so in this course.

One of the skills I seek to teach and assess is helping students to learn critical reading skills in reading primary and secondary texts. To that end creating ways to efficiently check their understanding and mastery of these more sophisticated skills has been a vexing problem for me.

That is the purpose of the example assessment I have created for this assignment- to assess students command of several specific critical reading skills in the context of social studies texts. These skills are specific to high school social studies standards in Michigan 

One of the assessment questions from my Assessment Design Checklist asks: does my assessment offer students a space to reflect on their learning of targeted skills and content? To that effort I have employed the Google Forms feature, Go To Section Based on Answer. This is a feature of multiple-choice test items. Depending on the answer a student selects in a multiple-choice item they can be sent to a specific section in the test. If a student selects the correct answer, they could be forwarded to a written response asking them to explain why they believe the answer they selected is correct. Conversely, students who select one of the distractors can be sent to a section that will ask them to reflect on their misconception and perhaps an additional reading to assist them in correcting their misconceptions.

I see this feature in Google Forms as a significant accordance because students will have the opportunity to be involved in assessing their own learning in thinking about why they got it right or wrong- thinking about their thinking.

My screencast below is a brief overview of this feature in Google Forms and how I am seeking to use this feature as leverage toward getting students to think about their own learning of critical reading skills.

 

Critical Review of Assessment Features of Google Classroom

For my critical review of a course management system (CMS) I have selected Google Classroom which I will refer to simply as Classroom. Much of my assessment will focus on Google Forms, otherwise known as Forms. Most of the assessment features in Classroom come through integrating Forms quizzes. While I have a bit more experience with Schoology and so far feel Schoology provides more flexibility and options than Classroom I now teach in a district where Classroom is the dominant CMS used by teachers. Because of this, I think it is essential for me to remain focused on assessing Classroom and Forms.

All the features I have assessed in Classroom and Forms are organized in their own tables where the name of the feature, the features description, affordances, constraints, and implications are explained. You can review my assessment of the assessment features of Google Classroom and Google Forms here.

Assessment Design Checklist 3.0

The creation of my FAD has been a series of snapshots of the evolution of my understanding of the concepts we have been considering and gaining a better grasp of the expectations for the FAD. My instructor, Sarah Frances Keena, has been patiently instrumental in pushing my thinking beyond the abstract and into the practical. She has frequently guided me through a greater understanding of what the course has required for the FAD. Specifically, she has encouraged me to flesh out more detail and be more specific with my rationale for the choices I have made in forming my FAD. Additionally, she has directed me back to readings I have already considered and as the course has proceeded I have been able to see some of the concepts with greater understanding. One example of this is parsing out the nuanced differences between students making meaning of what is being taught and students transferring their newfound understanding and skills to new and challenging context, Of course these two concepts are not wholly independent of one another which requires greater sensitivity on my part in making sure I see with clarity where I am assessing a student’s meaning-making versus their transferring of understanding and skills to new and different contexts.

I also received some of the most thought peer feedback I have ever experienced form, Jonathan Keck. His review of my work was a superlative example of constructive feedback. He understood what I was trying to accomplish with my FAD and demonstrated a thorough understanding of the educational context my FAD was designed for.

Here is a link to my ADC 3.0 for anyone who would like to review the work I have done thus far. Your feedback is more than welcome

Formative Assessment Design 2.0

For my pre-assessment instruction connected to my Formative Assessment 2.0 I am targeting three specific goals for my students:

1. Determine the meaning of a word or term in a historical text by using context clues
2. Identify the main idea in a historical text
3. Express their own understanding of the main idea of a historical text through writing a summary of the main idea in their own words

This brings to focus my instruction as I will use these goals of the FAD to drive the objectives of my lesson planning. We will spend one class period focusing using context clues to determine the meaning of unfamiliar words and terms. Next, we will focus on identifying the main ideas from a variety of texts. Finally, I will instruct students in how to summarize a text in their own words.

Students will receive some feedback within the formative assessment. My FAD 2.0 is designed in Google Forms. If students select the wrong answer they are directed to a constructed response test item which tasks them with identifying their own misconceptions and explain what they have done to correct their misconceptions.

The FAD 2.0 will provide me with more specific information regarding each individual student. The data can point me to specific students or groups of students that require more focused instruction on specific skills and knowledge.

Please explore or even take my quiz here.

Assessment Design Checklist 2.0

For my current iteration of the Assessment Design Checklist, I have added an additional question- question 4- inspired by our readings on feedback. The irony is that as of this writing I have yet to incorporate the great feedback I received for my ADC 1.0 questions- questions 1-3. Reading Hattie & Timperley has inspired me to start thinking about how I can design assessments that can provide instant feedback to the students and lead them through additional tasks to, as Hattie & Timperley put it, narrow the gap between current understanding and desired to understand.  I would propose that one could reappropriate that phrase in which, feedback helps to narrow the gap between current mastery and desired mastery. My currently unpolished version of my ADC 2.0 is available here.