Does This Open Questions or Close Them?
A Working Rubric for AI in Your Classroom
Two Students Your Syllabus Cannot Tell Apart
On a Tuesday in October, in a modern European history survey, two students sit three rows apart working on the same assignment: 1,200 words evaluating whether the alliance system or domestic politics carried more weight in the July Crisis of 1914, argued from the assigned primary sources. Maya opens a chatbot and types “what caused ww1 alliance system essay.” She receives four fluent paragraphs, moves the useful ones into her draft, smooths the seams, and submits an essay that cites none of the assigned documents and includes the phrase “a complex interplay of factors.” The question that was supposed to drive a week of reading got answered in eleven seconds, and she experienced the answer as relief. She has nothing left to ask after making her drive-thru academic order.
Daniel opens the same chatbot. He has already drafted his argument that the alliance system was the primary accelerant, and he types: “Attack this thesis as hard as you can. What would a historian who blames German domestic politics say?” The model hands him the Fischer thesis, the naval arms race, and the electoral pressures on the German government after 1912. He spends the next hour checking those claims against the assigned sources, concedes one point, and rebuilds his second section around the strongest objection. He leaves the session holding more questions than he brought to it, and his Wednesday is now organized around answering them.
Whatever AI policy your institution adopted last August treats these two students identically. A prohibition catches Daniel doing exactly the intellectual work the course exists to produce. A blanket permission blesses the encounter that just ended Maya’s thinking for the week. The debates that have consumed the last two years of faculty meetings, whether this counts as cheating, whether it erodes critical thinking, whether to standardize tools across the institution, all operate at the altitude of policy. Maya and Daniel get decided at the altitude of a single prompt. The question that works at that altitude is short enough to run between two raised hands: did this use of the tool open questions for this student, or close them?
The Damage Is Real, Measured, and Invisible Until the Exam
In 2024, a team at Wharton led by Hamsa Bastani ran the field experiment every faculty senate has been arguing in the absence of. Roughly a thousand high school students worked through math practice sessions under three conditions: a control group with no AI, an unrestricted “GPT Base” chat, and a “GPT Tutor” wrapped in guardrails designed to offer hints and withhold final answers.
The results expose a massive illusion of learning. The unrestricted group looked superb during practice, solving 48 percent more problems correctly. When researchers analyzed the chat logs, they found these students were using the tool as a crutch by overwhelmingly asking for and copying final answers. When the tool was removed for the exam, that same group scored 17 percent worse than the students who had never used AI at all (Bastani et al., “Generative AI Can Harm Learning,” 2024, DOI: 10.2139/ssrn.4895486). Strikingly, post-experiment surveys revealed these students had no idea they were learning less, as the fluency of the AI inflated their perception of their own competence.
Sit with the shape of that result. Every practice session felt like progress to the students, but the cognitive harm accumulated one closed question at a time. It stayed invisible until an unassisted exam made it legible. The guardrailed “GPT Tutor” group is the half of the study the policy debates keep missing. Forced to actively engage with hints rather than passively receive answers, those students posted a staggering 127 percent gain in practice performance and suffered no penalty on the final exam. The identical language model produced opposite learning outcomes depending on a single design variable, whether it resolved the students’ questions or held them open. The open/closed distinction stops being a metaphor at this point. It is an experimentally isolated switch, and somebody in your building is flipping it hundreds of times a day.
The mechanism behind the damage has been sitting in the learning-sciences literature for years. Manu Kapur’s productive-failure studies (Cognition and Instruction, 2008, DOI: 10.1080/07370000802212669) observed students who wrestled with ill-structured problems before receiving any formal instruction.
During the initial struggle phase, these students predictably underperformed compared to peers who received direct instruction first. However, on subsequent tests of conceptual understanding and knowledge transfer, the students who experienced the productive struggle significantly outperformed the direct-instruction group.
The discomfort of not-knowing that Maya dissolved in eleven seconds functions as a load-bearing wall in the architecture of learning. A tool that relieves that discomfort on demand, all semester, is quietly removing structure from a building that will be load-tested in December.
And there is a clock running. We know from the scholarship of teaching and learning that students calibrate to your posture on academic norms within the first three weeks of a term. Habits formed in September inevitably surface at grading time, when nothing can be redesigned. This identical pattern governs how students establish their expectations around AI use.
If you fail to intentionally set those expectations during that brief window, the structural damage to your course becomes obvious. You will know the problem has taken root in your classroom if any of these three symptoms sound familiar: your syllabus AI statement focuses entirely on authorized tools while ignoring cognitive processes; the most substantive thing a student can say about an AI-assisted answer is “it was helpful”; or you can no longer tell from written work which students actually did the reasoning. Because the window for establishing the right pattern is measured in weeks, it is cheap to act early and incredibly expensive to repair the damage later.
Building the Rubric from the Two Students
Hold Maya and Daniel in view and name the single difference between them. The tool was the same. The assignment was the same. The difference sits entirely in what each student was holding when the encounter ended. Daniel walked away holding a live question. Maya walked away holding a dead one.
Can we turn that insight into a reliable test you can run on your own classroom? The core premise requires asking a single master question: After this encounter with AI, does the student have more questions or fewer?
Everything else in this framework exists to make that master question observable. It has one serious, terminal flaw: you cannot see the inside of a student’s head. “More questions or fewer” is the correct metric, but it remains an invisible one. It requires practical instruments. It requires specific checks you can run at the exact moments in your week where you actually have the power and the room to act.
There are three distinct moments in a teaching cycle where you can intervene. Each moment requires a specific diagnostic question. Take them in the order your week takes them. Keep one of your own students in view throughout the process even as I create some examples for you.
1. At Design Time: Does the task require the student to evaluate, or only to receive?
When this occurs: This check happens before the students ever see the assignment. This is Sunday night at your kitchen table. This is syllabus week. This is the moment you are writing the instructions on the learning management system.
The Logic: Michelene Chi and Ruth Wylie’s ICAP framework (Educational Psychologist, 2014, DOI: 10.1080/00461520.2014.965823) compressed decades of engagement research into a hierarchy with teeth. The framework categorizes cognitive engagement into four distinct levels:
Passive Reception: The student merely receives information (e.g., listening to a lecture, reading a textbook, or reading an AI output).
Active Engagement: The student does something physical with the information (e.g., highlighting text, pausing a video, or copy-pasting an AI response).
Constructive Engagement: The student generates new understanding beyond what was presented (e.g., drawing a concept map, writing a summary in their own words, or critiquing an AI output against a rubric).
Interactive Engagement: The student constructively engages with a partner, defending ideas and negotiating meaning (e.g., debating a peer or actively prompting an AI to challenge their thesis).
The research proves the hierarchy operates linearly: interactive engagement outperforms constructive, constructive outperforms active, and passive reception loses to all three, across ages and across domains. Every AI encounter in your course lands somewhere on that hierarchy. Without intervention, the default landing spot is always the bottom.
Watch the biology student demonstrate passive reception. She types “explain cellular respiration in simple terms,” receives four clean paragraphs, pastes them into her notes, and closes the tab. This is passive reception with better production values than her textbook. Now, change exactly one sentence on your assignment sheet: “Submit the AI’s explanation along with a marked-up copy identifying one place it oversimplifies the process and one specific detail that Chapter 7 covers that the AI omits.”
She must now hold two accounts of the same process side by side and judge between them. She notices the chatbot never specifies where in the cell each stage occurs. She confirms this against the chapter. She writes the margin note. The output became the raw material for her judgment instead of a replacement for it.
The skeptical reader will immediately object that a student can simply ask the AI to generate the critique. They absolutely can. A subset of students will always seek the path of least resistance. We must abandon the fool’s errand of building uncheatable assignments. Our actual mandate is designing workflows where even the path of least resistance forces higher-order cognitive engagement. If the student pastes Chapter 7 into the chatbot and asks it to find its own omissions, they are still orchestrating a comparative analysis. They must read the output to find the missing detail, verify it makes sense in context, and transfer it to the margin note. Shifting the deliverable from initial production to critical evaluation traps the student into practicing constructive engagement. You are effectively causing them to learn despite themselves. That redesign cost you one sentence.
2. At Debrief: Can the student tell you why the output is good, incomplete, or wrong?
When this occurs: This check happens in the immediate aftermath of the work. In a physical classroom, this is the ninety seconds after class ends or a brief conversation in office hours. In an asynchronous online course, this is the required “process reflection” paragraph attached to their LMS submission, or the first reply in a threaded discussion. It is the moment the student hands in the work and you ask them to narrate their process.
The Logic: This is the only diagnostic you put to a student directly. A student who genuinely worked over an AI output can say something highly specific about its quality. They will tell you it skipped the proton gradient entirely. They will mention it was vague about where glycolysis happens. They will note they checked it against the lab manual and caught a hallucinated error.
A shrug, or a generic statement like “it was really helpful,” tells you the encounter was closed. What the shrug demonstrates is that the student answered the task exactly as it was designed. A failing answer to the debrief question serves as data about your assignment design, regardless of the student’s intentions.
The Intervention: The shrug demands a strictly pedagogical response. The student optimized for the path of least resistance your assignment permitted. Your immediate intervention serves as an act of modeling. You must show the student what rigorous, distributed cognition actually looks like. Whether you are standing in a physical classroom or replying in an asynchronous LMS thread, you provide the prompt they should have used. You instruct them: “Take this exact output, feed it back into the AI, and type: ‘Play the role of a skeptical biologist and identify the weakest link in your own explanation of glycolysis.’ Then tell me what the AI says.” You manually convert a closed encounter into an open one by demonstrating how to use the AI as a critical sparring partner. The structural intervention happens at your desk later that afternoon. You rewrite next week’s assignment to explicitly require this type of iterative, combative prompting.
3. At Unit Boundaries: Is the tool building a capability or replacing one?
When this occurs: This check never runs on a single encounter. This runs at the macro level, every three or four weeks. This is the end of a module, the week before a midterm, or the transition between major course themes.
The Logic: You must read patterns across a wider timeline. Some AI uses expose a student to cognitive moves they will eventually internalize as permanent habits of mind. We want Daniel to eventually run the “find the strongest objection” routine in his own head, completely independent of the software. If our ultimate goal is teaching students how to think deeply alongside these machines, our intervention cannot simply strip the tool away the moment they use it poorly. A punitive ban fails to teach the exact AI literacy they require.
When a student relies entirely on the AI to perform lower-order synthesis, the capability check requires them to orchestrate the AI to perform higher-order evaluation. You raise the cognitive floor of the assignment by requiring the student to submit their chat transcript. You grade them specifically on the architectural quality of their iterative prompting. Did they instruct the AI to generate counter-arguments? Did they force the AI to reconcile conflicting primary sources? The repair is a stretch of rigorous practice where the student is explicitly evaluated on their ability to direct the AI’s cognitive labor.
This repair orchestrates the student’s evolution into a manager of information. To use the language of the ICAP framework, you are structurally mandating the use of AI for constructive engagement. The student must generate new understanding beyond what the model initially presented to pass the assignment. Even if a specific task does not reach the pinnacle of interactive engagement—where the student and the AI truly negotiate meaning as peers—securing a baseline of constructive engagement ensures the tool actively builds cognitive capability.
Notice what all three diagnostic questions share. Each one focuses relentlessly on the student’s cognitive state. None of them focus on the mechanics of the tool itself. That design choice is deliberate, and it leads exactly to where the rubric gets harder to apply.
The Same Encounter Scores Differently for Different Students
A senior physics major with strong preparation asks a model to probe the gaps in her understanding of quantum mechanics. She can judge what comes back, push on it, and treat the system as a sparring partner; the four questions pass almost automatically because her stance toward the material was already open. An anxious sophomore in the same course, holding a shakier foundation and a purely instrumental relationship to the grade, wants one thing from the identical tool: relief from the discomfort of not knowing. That difference was in the room before the tool arrived. The tool amplifies it, on both ends, every day.
The relief is the trap, and there is a measured reason to distrust it. Louis Deslauriers and colleagues at Harvard (PNAS, 2019, DOI: 10.1073/pnas.1821936116) taught identical introductory physics content two ways, polished lecture and active learning, in a randomized crossover design. Students in the active sessions learned measurably more and reported learning less; the fluency of an expert delivery inflated their feeling of learning while the actual learning lagged behind it. A chatbot is the most fluent explainer ever placed in front of a struggling student. To the anxious sophomore, the closed encounter feels like the most productive studying she has done all term, and Deslauriers’s data says that feeling runs opposite to the fact.
So the rubric cannot be run from the syllabus alone. It has to be run at the level of this student, in this encounter, with this relationship to the material, which requires knowing where each student stands relative to the content, the course, and their own confidence as a thinker. That requirement is why the rubric cannot be automated, delegated to a policy document, or outsourced to a detection tool. Read that limitation as its defense of your profession: the moment a framework for good teaching can be applied without a teacher’s judgment is the moment it has stopped being about good teaching.
The Pedagogical Turn
This brings us to the final phase of the framework. You will notice that the remaining sections—evaluating your own lectures, identifying institutional failure modes, and executing the ten-minute diagnostic—step away from the mechanics of artificial intelligence entirely. This shift is deliberate. We are no longer discussing software; we are discussing foundational pedagogy. AI is merely the diagnostic dye injected into our educational systems, illuminating exactly where our instructional designs were already relying on superficial compliance. The technology will update every six months, but the underlying teaching posture required to manage it remains permanent. To wield this design, debrief, and boundaries AI rubric effectively, you must turn it away from the machine and point it at the classroom itself.
Run It on Your Own Teaching First at Design
I fail my own rubric. For years I designed seminars and assignments I would have described as open inquiry, and by the standard of the master question, some of them were theater. I asked open-sounding questions while holding a mapped destination, and I counted the design successful when the students arrived at it. Students are never naive about this. They feel the rails under an assignment, and once they do, they stop exploring and start guessing the answer in the teacher’s head, because guessing is what the room actually rewards. That is a closed encounter with better manners: the form of open inquiry wrapped around a predetermined terminus, which is the exact same failure I have spent four sections attributing to a chatbot.
If that is what we count as learning, then as a teacher I can be replaced.
Before you apply the diagnostic questions to your students’ AI use, run them on your own course design. When you build a prompt, are you willing to be surprised? Are you willing to let the student’s research land somewhere that is missing from your rubric? If your honest answer is that your assignments have known answers and the students’ job is to find their way to them, AI has handed you a mirror at an uncomfortable resolution. The sting is worth keeping, because a teacher who has felt the difference between performing inquiry and practicing it starts seeing that difference in their own syllabus, and that is where the redesign begins.
Where This Breaks at the Debrief
The debrief is the most critical and fragile moment in this framework. It is the only time you run the diagnostic directly on the student, moving the focus away from the assignment sheet. When you ask a student to narrate their AI process, you are opening a conversation about cognitive labor that most students have been trained to hide.
Three failure modes show up reliably when educators attempt to execute this debrief, and avoiding them is the playbook for making the framework actually function:
The Debrief Curdles into Interrogation: The moment a teacher asks about AI shortcomings in a punitive tone, students take their tool use underground. A hidden encounter can never be evaluated or guided. The framework only functions in daylight. Your primary job is keeping the lights on by grading the critical evaluation as the actual work.
Evaluating the Software: During the debrief, educators inevitably attempt to diagnose the specific platform, ignoring the student workflow entirely. Concluding that a specific large language model inherently closes questions simply reproduces blanket prohibition using new vocabulary. Applied honestly, the exact same tool will pass or fail within a single class period depending entirely on how the student orchestrates it.
Inquiry Theater: When generating questions becomes a mandatory deliverable of the debrief, students quickly learn to manufacture curiosity on demand. This is a closed encounter wearing an open costume. To fix this, make the curiosity load-bearing: whatever a student claims to wonder about becomes the required starting point for their next assignment. Questions written without consequence get written without thought.
The First Ten Minutes for Unit Boundaries
The unit boundary check is designed to measure patterns over a month, but establishing that baseline begins with a single diagnostic. If you are overloaded and skeptical, skip adopting the entire framework and run one micro-test to establish your baseline today.
Tomorrow, pick a recent assignment where you suspect AI absorbed the cognitive labor. Choose two students. Ask each of them to pull up their output and find one specific error or omission in the model’s logic.
Two substantive critiques mean your unit design already produces open encounters, providing you with concrete evidence of student engagement. Two shrugs mean you have found your first macro redesign target. Either way, you will know something vital about the boundaries of your classroom that no institutional policy document can tell you, and it will have cost you ten minutes.
The Master Question
We began this essay with two students your syllabus could not tell apart in terms of AI use if all we are concerned with is what tool and when it should be used. You now possess the instrument that distinguishes them, a pedagogical posture that models AI co-piloting, and a structural repair for every shrug the classroom returns.
Artificial intelligence will continue to scale the production of fluent text, driving the cost of a static answer to zero. As the value of an answer plummets, the value of a live question becomes paramount. The discomfort of not-knowing remains the load-bearing wall of human cognition, and our mandate is to design environments where the machine actively reinforces that structure.
The institutional policy debates will still be grinding in May of 2027. Who will be a Maya and a Daniel in your classes this Fall will largely get decided by the decisions you are making this summer and implement by mid September at the latest. So post the master question on your wall and ask yourself after every assignment redesign you are mulling over:
After this encounter with the AI tool, does the student hold more questions, or fewer?