Sight Reading Comprehension Assessment

Often I have struggled with how to give a true reading comprehension assessment, asking myself, "Should it be over a known, familiar passage? Should it be a seen passage? How many questions should it be? How long should the passage be? What exactly am I assessing in reading comprehension - what are the skills I am wanting to see in students when they read?" 

I have been a coach for Martina Bex and Elicia Cardenas' Acquisition Boot Camp (ABC), and the final week of lessons addresses assessments and various ways to evaluate presentational, interpretive, and interpersonal modes. One of the assessments discussed is reading comprehension, and I absolutely love how they present it, because it addressed all of my concerns.

Essentially, you give students a sight passage in the target language to read, but this passage uses known, targeted vocabulary/cognates with some unknown words glossed. Students will answer questions about the passage IN ENGLISH in 5 categories: 

• Main Idea - What happens? What is the author telling us?
• Details - Find three details which support the main idea.
• Translation - Translate the following sentences into English
• Inferences - Based on the context of X sentence, what do you think that this unknown word means? Since X word is a cognate, what do you think that it means? 
• Drawing Conclusions - Why do you think that X person does (Y thing) - support with details from the story.

Recently, I gave my Latin 2 classes a sight reading comprehension assessment based on the above. We are currently reading Emma Vanderpool's novella Incitatus, so I wrote up a passage which incorporated the targeted vocabulary and lots of similar sentences (if not identical) to the Incitatus chapter which we had just finished.

I only did three of the categories (main idea, details, drawing conclusions) for the assessment, but I like how Martina and Elicia have set up the categories.

Observations

1. I was unsure how my Latin 2's would do on this, considering that this group of Latin 2's has been more like "Latin 1.5" in this post-hybrid educational setting (and there is nothing wrong with that, considering how things were last year!). As a result, I feel like I have been taking things much slower and really sheltering vocabulary (and not grammar) with them. However, based on student scores, outside of a few outliers, most students scored above 90. And for ​those who scored below a 70, it shows me that they need more work with the language. 
2. Why do reading comprehension assessments in English and not in the target language? I think Martina Bex says it best here. Years ago, I used to assess reading comprehension in the target language, but after reading Martina's post, I immediately switched to English. 
3. Why do a sight passage instead of an already-seen, familiar passage? A sight passage using known targeted vocabulary and some glossed words will let me know what students are understanding this semester based on something which they are reading for the first time using the language which we have been targeting thus far; therefore, it is VERY important that the passage is understandable and comprehensible for students. If I were to use an already-seen, familiar passage, then I am not truly assessing their reading skills - it ends up being a memory recall of what we have discussed and gone over in class, even if I were to include the text on the assessment. In other words, for many students it would not require them to even look at the passage at all, because they already know the answers. That is not the purpose of a reading comprehension assessment.
4. I think formatives are good for already-seen, known passages, but I definitely like this format for summatives.
5. I love the Inferences and Drawing Conclusions sections, because those are great ways to ask higher-order thinking questions based on a reading.