Role Play Sample

Colleen is a Sociology faculty member.

Marty is a Computer Science faculty member.

The context is that Marty is moving one or more of his courses to an online format.

Marty: Colleen, I am struggling to find a way to do exams in my classes now that we are doing things all online. My exams are such a big part of my evaluation scheme for students. It gives me a real sense of what they know and what they can do. I have looked at the lockdown browser options that the university has given us, but they don’t meet my needs. I still can’t be sure that students are doing their own work.

Colleen: Tell me more about what you are hoping to get out of the exam evaluation process and what the specific issues are with the lockdown browser options. Is this a software issue or a faculty exam framing issue?  What are the specific gaps and what would they allow students to do that would be unethical?

Marty: I am hoping that I can get a sense of what each student can and cannot do. I want to know if they understand the vocabulary words and the basic concepts we are addressing in the class. I guess I feel a responsibility to report to the rest of the world what students have learned in my class, and I don’t feel I can do that—at least honestly—if I don’t know that students are doing their own work.

And the lock down browsers don’t help with that problem. Students have multiple devices—at least many of them do—and they can “get help” from the rest of the internet. If I were to use lockdown browsers, I think it would only create new problems: some students will be playing by the rules and others not. I won’t be able to tell them apart and now I think I have an even bigger issue: I give an honest reporting of what some students have mastered and a dishonest reporting on the rest. The worst part is, I don’t know which is which!

I guess I see this as a software issue: the lockdown browser pitches itself as a secure way to give an exam—leading me to think it is at least as secure as when I give an exam in class. That sort of claim makes me uncomfortable.

Colleen: Yes, I see.  So, it is really a multi-edged issue, not just are the students being honest, but when relying on a device like a lock down browser, whether in the end you can do a fair assessment and report out honestly.  So, a couple of additional questions.  First, can a lock down browser still be used, but with another requirement at the beginning that requires students to sign a statement indicating that they will not use other devices.  In other words, do you think that requiring a “pledge” would help the situation?  Second, with the lock down browser as you are describing it, is it “timed” in a manner that helps to prevent the use of other devices in a timely enough way to discourage cheating?  Finally, I don’t think I know enough about how you’ve constructed the “test–in other words, are you having them create a narrative based on parameters that you’ve determined to be valuable for assessment?  In many of my courses to test vocabulary and concepts, I construct essay questions that require students to demonstrate ability without an easy use of existing materials from the internet.  This would, of course, require you to engage in ongoing reconstruction of tests/quizzes.

Marty: I could ask them to sign a statement, but that doesn’t sit real well with me either. Overall, I trust most of my students and such a pledge sends a signal to the contrary. If there is a bad apple out there, I doubt that such a pledge is going to prevent them from looking up answers or getting help from somewhere out there on the internet. I think most of my questions are a lot like you describe, so I don’t think I have to restructure much—other than how I think about this. It’s also the case that half my exam is problem solving—they have to show their work, and I grade the work that they do, not merely the answer.

Since I trust my students, I think I am going to just tell them that. I’ll use Zoom to watch them while they take the exam rather than use lockdown browsers. Then I can avoid all the issues with them. I am going to tell them that yes, the environment allows them to use outside resources, and I hope that they don’t. I’ll invite them to see the exam a little differently—more of a chance to share with me what they know and can do in the given amount of time. I know there are issues with time constraints on exams, but I think that is an issue for another time.

Colleen: There are some other conversations we can have. Do exams really tell you what students know and what they can do?  In many ways exams with memorization schemes and as you suggested, short response times are counterproductive to understanding how people are approaching a “problem” and what interactive approaches they might use to solve it.

This role play was developed by Colleen Greer and Marty J. Wolf as part of the Mozilla Foundation Responsible Computer Science Challenge. These works are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.