{"id":552,"date":"2026-08-27T21:00:38","date_gmt":"2026-08-28T01:00:38","guid":{"rendered":"https:\/\/solutionsreview.com\/edtech\/?p=552"},"modified":"2026-08-27T21:01:25","modified_gmt":"2026-08-28T01:01:25","slug":"test-what-students-can-do-without-the-tool","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/edtech\/2026\/08\/27\/test-what-students-can-do-without-the-tool\/","title":{"rendered":"Test What Students Can Do Without the Tool"},"content":{"rendered":"
\n
\n
Meryll Dindin offers insights on testing what students can do without “the tool.” <\/strong>This article originally appeared in Insight Jam<\/a><\/em>, an enterprise IT community that enables human conversation on AI.<\/span><\/strong><\/div>\n<\/div>\n<\/div>\n

\"\"<\/a><\/span><\/p>\n

I run technology at a K-12 teletherapy company that places licensed special-education clinicians with students across 27 states, and I spent this summer reading the research on what artificial intelligence actually does to learning. Both jobs keep returning me to the same question, and it is one I rarely hear asked in a procurement conversation: What can a student do once the software is taken away?<\/p>\n

American school districts accessed an average of 2,982 distinct digital tools during the 2024-25 school year, per\u00a0LearnPlatform by Instructure’s eighth EdTech Top 40 report<\/u><\/a>, up nearly 9 percent year over year. Nearly all of them now carry an AI feature. Adoption has stopped telling a buyer anything, because everything has been adopted. What separates these products is whether the software finishes the work or hands it back to the student.<\/p>\n

What 1 Field Experiment Settled For Me<\/strong><\/h4>\n

Hamsa Bastani and colleagues put a number on that question in the\u00a0Proceedings of the National Academy of Sciences<\/u><\/a>\u00a0last June. They gave nearly a thousand students in grades nine through 11 at a large Turkish high school access to a GPT-4 tutor during math practice. One group used an interface that behaved like standard ChatGPT. Another used a version prompted to withhold answers and walk students through teacher-designed hints.<\/p>\n

While the tool was in hand, both groups improved: 48 percent better practice grades for the plain interface, 127 percent for the guided one. Then the researchers took the tool away and ran an exam. Students who had used the plain interface scored 17 percent worse than classmates who never had access at all. Students who had used the guided version showed no penalty.<\/p>\n

Both arms ran the same underlying model and produced opposite outcomes on the unaided exam. I have sat through enough vendor demos to know that this never appears on a slide. The variable was a product decision about when the software is allowed to answer.<\/p>\n

Policy is Not Going to Make This Call for You<\/strong><\/h3>\n

Education AI policy moved faster than the evidence and landed somewhere other than learning. New York City banned ChatGPT in January 2023 and reversed itself by May. Ohio now requires every public school district, community school, and science, technology, engineering, and mathematics (STEM) school to\u00a0adopt an AI policy by July 1, 2026<\/u><\/a>, the first state mandate of its kind, enacted through House Bill 96. The statute sets the requirement without specifying what the policy must contain, and the state’s model policy is optional.<\/p>\n

Nearly all of this rulemaking governs access and student data. The exceptions that reach learning are assessment decisions made after the fact. England’s qualifications regulator, Ofqual, published its position in January 2026 that\u00a0AI may not be the sole basis for a mark<\/u><\/a>\u00a0in a regulated qualification, and Princeton’s faculty voted in May 2026 to\u00a0proctor in-person exams<\/u><\/a>, ending 133 years of an unproctored honor system that began in 1893.<\/p>\n

Both are containment measures written for institutions that already bought the tools, and neither asks a district to establish whether a product scaffolds a student or substitutes for one.<\/p>\n

What District Buyers Tell Me They’s Weighing<\/strong><\/h3>\n

Districts have become harder buyers over the past two years. In the latest EdTech Top 40, 45 percent of tools carry published Every Student Succeeds Act (ESSA) research, up from 32 percent the year before. Usage-metric pitches (logins, minutes, seats activated) increasingly get rejected outright, and outcomes-based contracts that put a fifth to a half of vendor payment at risk have started to appear.<\/p>\n

In my own customer interviews this year, user-friendliness came back as a deciding factor after several years of losing ground to feature breadth. Special-education directors on our advisor panel rank their criteria in a consistent order: cost first, then evidence of progress over time, then whether the tool still helps the classroom teacher after the session ends. A product that adds training burden to a stretched teacher is an adoption risk even when its clinical case is strong.<\/p>\n

None of that tests the thing the Turkish experiment isolated. Efficacy evidence almost always measures outcomes with the tool present. It answers whether performance improved, which is the exact measurement that study showed can move in the opposite direction from learning. A pilot can test for the other direction with three changes:<\/p>\n