The Three Things I Believe About Assessment: Revised

The first act in CEP813 was to write about three things I believe about assessment. This is a great approach to set up a student to have an anchor point to reflect on how their concepts, assumptions, and beliefs have been transformed by learning new concepts. This is certainly the case with my experience in CEP813.

My first stated belief at the beginning of the course was that multiple-choice questions are virtually worthless. Of my three stated beliefs, this is the viewpoint that has shifted the most. As I wended my way through the course, Toward the beginning of the course my notions regarding multiple-choice questions were put to the test. In Module 1 we considered the text, Knowing What Students Know, published by the National Academy of Sciences. The authors of Knowing What Students Know assert that assessment should rest on three pillars, one of which is “tasks or situations that allow one to observe students’ performance” (National Science Foundation, 2001, p.23). This led me to wonder, can a multiple-choice question satisfy the requirement of a teacher observing a particular task to elicit a particular performance task that provides the teacher with an essential understanding of a student’s proficiency? The conclusion I arrived at was that multiple-choice questions could play a constructive role in doing so. How so? Certainly multiple-choice questions on their own are extraordinarily limited as tools of assessment, however, they can serve as a doorway toward deeper thinking and reflection.

This insight led me to use multiple-choice questions as a quick check of a students grasp of a particular skill. For example, having students read a passage of text and then taking them to use context clues to determine the meaning of a particular word turned into a doorway where, if the student demonstrates a lack of proficiency in the skill, they could write a metacognitive reflection as to why they had the misconception and what they did to correct it.

My second stated belief was that assessment should be integrated as a part of how students learn. While what I learned in CEP813 affirmed this position the course changed the core reason’s I believed so. Chief among these was our consideration of the importance of feedback. In my reading of Hattie and Timperley, I learned that “Too often, assessments are used to provide snapshots of learning rather than providing information that can be used by students or their teachers” (Hattie. Timperley, 2007, p. 104). This made me rethink what it meant for an assessment to be an integrated part of assessment As a consequence I explored ways I could provide students immediate feedback on my assessment designs and I discovered that technologies such as Google Forms afforded me the ability to guide students through a digital assessment embedded with automated feedback contingent upon their responses to various test items.

My third belief was that assessment should be designed to provide effective feedback to students. As you can see, I now see effective feedback and assessment as a learning activity as an integrated aspect of effective assessment. Helping students gain mastery of the language and skills specific to the discipline being assessed, in my case history, is essential to student proficiency. I thought I understood James Paul Gee’s concept of the Semiotic Domain but it was not until I created my game based Assessment that I truly appreciated what he was proposing (Gee, 2001). Formulating the game-based assessment engaged me in thinking more concretely about the role procedural language in assessment and how such an assessment can not only measure the degree to which a student has gained command of the semiotic domain of a particular disciple but also can serve as an assessment for learning at the same time.

References

James Paul Gee, Schallert, D. L., Hoffman, J. V., Maloch, B. E., Worthy, J. E., & Fairbanks, C. M. (n.d.). 51St Yearbook of the National Reading Conference (San Antonio, Texas, December 5-8, 2001)., pp. 23-32 Publication year: 2001

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. Retrieved from http://www.jstor.org.proxy2.cl.msu.edu/stable/4624888

National Research Council. 2001. Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: The National Academies Press. https://doi.org/10.17226/10019.

Formative Assessment Design 3.0

For my pre-assessment instruction I am targeting three specific goals for my students:

1. CONTEXT CLUES: Determine the meaning of a word or term in a historical text by using context clues.
2. THE MAIN IDEA: Identify the main idea in a historical text
3. SUMMARIZE: Express their own understanding of the main idea of a historical text through writing a summary of the main idea in their own words

This will bring focus to my instruction because these goals of the FAD will drive the objectives of my lesson planning. Embedded in both my instruction and FAD 3.0 is an emphasis on promoting peer to peer and student to teacher dialogue as an essential part of how students receive and process feedback (Nicole, McFarland-Dick, 2006, p.209).  

CONTEXT CLUES

• For pre-assessment instruction, I will model using context clues to determine the meaning of a specific word. Before instruction, I will select a specific word that is not common in high school vocabulary. During instruction, I will identify the word I wish to understand the meaning of and then do a think aloud as I read the selected text. A think-aloud is writing down and verbally explaining my thinking process as I read. As I read, I will describe how the surrounding sentences and text lead me to the definition of a particular word.
• Following my modeling activity, students will work in pairs to perform the same context clue task with a different word provided to them. Once the pairs have completed the task they will compare their results to the other pair of students at their tables identifying what clues they had in common and what clues they did not.
• In my FAD 3.0, students will perform the same task except in the FAD 3.0 students will select a statement that best matches what they believe the definition of the word is based on their context clues analysis. Whether the students select the correct answer or not, they must explain the context clues they identified and used to determine the correct answer or the context clues they used in a second attempt at determining the correct answer.
• With the assessment complete there are two specific ways I will use the resulting data. First, students will compare their results with one of their Compass Partners. Compass partners are a set of peers that each student selects as their preferred fellow students for sharing and peer-reviewing each other’s work for peer feedback. 
• Because my FAD 3.0 is a Google Form, I have access to instant results of the multiple-choice items which will allow me to identify students I wish to work with during the peer review process. As students work with one another in their peer review and feedback I will circulate among the students, focusing on students whose assessment scores suggest they need supplemental instruction and support. As I work with different Compass Partner pairings I will be listening carefully to the peer feedback. Depending on the content of the peer feedback I may ask further questions to push student thinking or offer further direct instruction to target areas of demonstrated need.
• Secondly, I will analyze the results of the FAD 3.0 to determine if certain questions sets were problematic for most students across class sections and groups of students. If I determine that there were questions that had a high percentage of error I will review both the structure of the question itself to identify whether the error percentages were due to a poorly written question or if further whole class or group supplementary instruction is required.

THE MAIN IDEA

• For pre-assessment instruction of identifying the main idea, I will use the think-aloud approach again. In this case, I will obviously think aloud my reading and explain the critical reading strategies I am using in identifying the main idea of a text. The main strategy will be to prioritize the importance of sentences within the text from the most essential to the least. In my modeling, I will select what I believe are the three most important sentences or passages of text and explain why I believe so.The most essential sentence or passage of text is the one that best expresses the main idea of the text entire text.
• Following my modeling activity, students will work in pairs to perform the same task of identifying and prioritizing the three sentences or passages of text that are most essential to the main idea of the paragraph. Of course, students will be provided a different text than the one I used to model the task. Once the pairs have completed the task they will compare their results to the other pair of students at their tables identifying what three sentences or passages of text are most essential to the main idea as well as prioritizing from 1-3 sentences or passages of text most essential to the main idea.
• In my FAD 3.0, students will perform the same task except in the FAD 3.0 students will select a sentence that best represents what they believe the main idea of the text is.
• With the assessment complete there are two specific ways I will use the resulting data. First, students will compare their results with one of their Compass Partners. Compass partners will compare their results and examine and explain to one another why they think they selected the correct or incorrect answer.I will use the results of the FAD 3.0  to identify individual or small groups of students who I wish to provide follow-up instruction. As the students compare and analyze the reasons behind their choices analyze I will offer questions to push student thinking or provide further direct instruction students I have identified as requiring further instruction and support.
• Secondly, I will analyze the results of the FAD 3.0 to determine if the question was problematic for most students across class sections and groups of students. If I determine that the question had a high percentage of error I will review both the structure of the question itself to identify whether the error percentages were due to a poorly written question or if further whole class or group supplementary instruction is required. 

SUMMARIZE

• For pre-assessment instruction of summarizing the main idea I will again use the think aloud approach.Using the same text I used to identify and prioritize the three most essential sentences or passages of text to the main idea, I will model by thinking aloud how to identify words essential to understanding the text from the top three sentences. Following that I will model how to reformulate the language into my own words as a model for the students how they can best summarize the main idea in 3-5 sentences.
• Following my modeling activity, students will work individually— unless paired with a partner in certain exceptional cases— to perform the same task of identifying key words and terms and writing a summary of the main idea in their own words. Of course, students will be provided a different text than the one I used to model the task. Additionally students will be provided a “Summarizing the Main Idea” rubric. Once students have completed the task they will compare their results to their shoulder partner at their table for peer review and peer feedback. Peers will use the “Summarizing the Main Idea” rubric in order to provide feedback. I will circulate among students to ensure students remain on task and support students as I see the need.
• In my FAD 3.0, students will perform the same task of summarizing the main idea. With the assessment complete there are two specific ways I will use the resulting data. First, students will compare their summary of the main idea with one of their Compass Partners. Additionally, compass partners will provide feed-back to their peers using the  “Summarizing the Main Idea” rubric. As the students compare and analyze summaries, I will offer questions to push student thinking or provide further direct instruction students I have identified as requiring further instruction and support.
• Students then will have the opportunity to revise their summaries by incorporating the feed-back from peers and myself based on the  “Summarizing the Main Idea” rubric. 
• I will score all summaries with the  “Summarizing the Main Idea” rubric. If I find that a large percentage of students received a low rubric score on one specific criteria I will engage students in further instruction and reflection on that particular criteria and create space for meaningful peer to peer and student to teacher dialogue based on student reflections on their understanding and mastery of that particular rubric criteria.

Please explore or even take my quiz here

References

Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/0307507060057209

Embedded in a Medieval French Village During the Black Plague

 

In CEP813 we have had the opportunity to be introduced to a simple game-based technology called Twine which can be found at twinery.com. With this platform, you can provide students with a game-like activity in which they can make choices that lead them through various activities that address their instructional needs for mastery of knowledge and/or skills. I had the pleasure of playing one of my peer’s games who happens to be a history teacher as well. For their game, the player is essentially embedded in a small village in Medieval France as an elder in the community. The village faces the specter of the Black Death as it sweeps across the southern regions of France. As a village elder, the player is faced with various scenarios. For example, whether to read a letter from the King or an itinerant priest. Then the player must decide what to do based on what they read in the letters. Depending on those choices determines the fate of the village.

Because I am too, a history teacher I expected to perform well in the game. I expected that as I played the game the outcomes would clearly demonstrate to me how I performed. As I played the game, I understood where the teacher wished to take players. Because much of the Medieval belief system was based on superstition, many of the choices students faced were grounded in those superstitions. The point of the game became clear toward the end of the game. Ignoring the King’s command to arrest and kill the itinerant priest led to the village’s survival whereas choosing to arrest and hang the priest led to the demise of the village within two weeks.

While the game was fairly easy for me to understand it holds promise in captivating students as they attempt to navigate a very unfamiliar landscape. I predict many students will make choices that lead to the demise of the village and this will be an engaging surprise for them.

As an assessment for learning, this particular game accomplishes that goal. That is the cool thing about making a twine- students have the opportunity to go back, reflect, and rethink their thinking process if they went down an undesirable path. The only suggestion I would make regarding this game is perhaps students could be alerted to the fact that the fate of the village rests in their own hands. That way the students understand that the survival of the village is the signal of their success in playing the game.

The Game: 2.0

The game I have designed in Twine seeks to reinforce the internal and external grammar of historians. It will reinforce internal grammar by tasking students to engage in specific historical thinking tasks to successfully navigate their way through the game. This means they must pay careful attention

One of the skills that students focus on mastering in my classroom is historical-critical reading skills such as corroboration of evidence and sources, analyzing the historical context of sources, vetting the credibility of a secondary source, and determining the bias and frame of reference from a primary or secondary source.

The twine game will be set up to help students to either corroborate multiple documents and/or determine if they are an appropriate source for responding to a historical problem.

The game will seek to reinforce the internal and external grammar of historians. It will reinforce internal grammar by tasking students to engage in specific historical thinking tasks to successfully navigate their way through the game. For example, for internal grammar, exposing students to content that historians would deem appropriate for the historical practice of interpreting the past, namely, primary source documents.

When students navigate through the game they must successfully determine which side of a historical argument a particular primary source would support. Students, then, must engage with internal grammar, eg. primary sources, historians deal within their work. This is the essence of the external grammar of historians- that they allow all the evidence lead them to their interpretation and conclusion by means of corroboration of all the available primary source evidence

• Students will be presented with the following historical problem:

Should President Harry Truman have dropped the atomic bombs on Japan or was this the wrong decision?

• Students will answer the question before weaving their way through a series of documents related to the decision to drop the atomic bombs on Japan.
• For each document, students will decide whether the United States should or should not have dropped the bomb.
• Based on their choice, students will then consider a document that is strong evidence against their previous choice. An example of this is Harry Truman’s explanation for dropping the bomb. This source most obviously supports the position that the dropping of the atomic bomb was necessary. If a student were to make the choice that Truman’s rationale for dropping the bomb was necessary then they would move on to Admiral William E. Leahy’s memoir in which he states that the bomb did not contribute any advantage to defeating Japan. This document supports the argument that the U.S. should not have dropped the atomic bomb. If the student, however, chose to use the document to support the dropping of the bomb they would be exposed to another document that also is evidence against dropping the bomb thus providing the student an opportunity to evaluate a document that does not support Truman’s decision
• At the end of the Twine, students will be tasked to take a final position to the question and then collect their evidence from the Twine activity that corroborates with one another and supports their position. For example, students will be presented with all the primary source readings in the twine and then take a position as to whether the bomb should have drooped or not and select the primary sources that best supports their argument. This final task in the twine is what would be used to assess student’s command of evaluating primary source documents, analyze for bias, and corroborate supporting primary source evidence.

So how will the assessment meet the expectations of Assessment Design Checklist?

---

1. QUESTION: Does my assessment task students to express how they have made meaning of the knowledge and concepts?

EVIDENCE: My assessment tasks students to make practical application of knowledge, theories, and concepts to novel problems

This assessment provides students the opportunity to make meaning of the content (internal grammar) through using historical thinking skills to make their own independent position regarding the historical problem presented in the assessment. 

---

2. QUESTION: Does my assessment task students to apply thinking skills in a manner where they must transfer their knowledge of concepts to new and/or unfamiliar contexts?

EVIDENCE: My assessment tasks students to apply their understanding of learned concepts by synthesizing, analyzing, and evaluating problems embedded in new and/or novel contexts.

While this is, obviously, an old historical problem that historians have grappled with for decades it is a novel historical problem for the vast majority of students. To make meaning of this content they must apply, or transfer, the historical thinking skills they have learned in a new and novel context.

---

3. QUESTION: Does my assessment aide in reducing the discrepancy between current and desired understanding?

EVIDENCE: Feedback provided to the student from the assessment helps the student improve performance in the targeted task in a measurable manner.

This assessment accomplishes this goal by allowing students to work independently by applying historical thinking skills to content. When they evaluate a primary source incorrectly, they are directed to another document which possesses the same bias thus providing the student with the opportunity to identify their own discrepancy. 

---

4. QUESTION: Does my assessment offer students a space to reflect on their learning of targeted skills and content?

EVIDENCE: Students write reflections identifying how they corrected misconceptions. Students will specifically write about their understanding and mastery of targeted content and skills.

For the final task, students take their own position on whether the US should have dropped the atomic bombs on Japan. The prompt requires them to select two of the sources they believe supports their position. They will have to explain how each of the sources they select supports their claim. Once they submit their writing I will review their work and determine whether the sources they selected to support their position. Once I have evaluated students’ responses I can determine which students have gained mastery and which students require further opportunities to master the skill. What I choose to do instructional will depend on how many students demonstrate mastery versus how many students require further practice.

Based on my previous experience with similar exercises I predict that enough students will need further instruction and time to practice the skill. In that event, I will assign students who have gained mastery to formulate an argument opposite of the one the made for their initial claim.

Take my Twine for a spin