{"id":638,"date":"2026-10-15T10:28:25","date_gmt":"2026-10-15T14:28:25","guid":{"rendered":"https:\/\/solutionsreview.com\/edtech\/?p=638"},"modified":"2026-10-15T11:06:26","modified_gmt":"2026-10-15T15:06:26","slug":"grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/","title":{"rendered":"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates"},"content":{"rendered":"
\n
\n
TalentAid AG’s Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated. <\/strong>This article originally appeared in Insight Jam<\/a><\/em>, an enterprise IT community that enables human conversation on AI.<\/span><\/strong><\/div>\n<\/div>\n<\/div>\n

\"\"<\/a><\/span><\/p>\n

In the summer of 2023, researchers at the University of Reading in England tested their own university\u2019s exams. Through 33 fake student accounts, they submitted answers written entirely by an artificial intelligence (AI) chatbot<\/a> to take-home exams in five undergraduate psychology modules. None of the markers knew. The markers flagged 6 percent of the AI submissions, and in four of the five modules the AI answers earned higher grades on average than the real students\u2019 work.<\/p>\n

A finished answer used to be fair evidence that the person who handed it in could do the work. Once a tool can produce a passable one on request, grading the answer tells a teacher or an employer very little. If I were redesigning assessment, I would hand people a plausible piece of work with something wrong in it. Then I would grade how well they find the flaw and defend the finding.<\/p>\n

What the practice scores hid<\/strong><\/p>\n

A randomized trial published in the Proceedings of the National Academy of Sciences<\/a> in 2025 shows how far the work and the learning can drift apart. Hamsa Bastani and colleagues worked with nearly 1,000 mathematics students at a high school in Turkey. Every student practiced with textbooks and notes, and some classes also had one of two AI tutors. Those given a version that worked like a standard chatbot scored 48 percent higher on the practice problems than students with no tutor. On the closed-book exam that followed, without the tool, they scored 17 percent lower than students who never had it.<\/p>\n

In the researchers\u2019 own tests, that chatbot gave a correct answer to the practice problems only 51 percent of the time. The authors suggest students were \u201ceither unable to detect these failures or unwilling to spend the effort needed to check correctness.\u201d The students using it did not perceive that they had learned less. A second version, built to give hints and withhold answers, largely removed the exam penalty.<\/p>\n

What I see in engineering<\/strong><\/p>\n

At TalentAid, roughly 95 percent of new code is written by AI. Our process got stricter over the same period. Every change now passes a mandatory human review plus automated checks, where before most changes were merged by their author within minutes. I used to average about three pull requests a week. Now it is around 15, at the same or better quality.<\/p>\n

A review is a critique. The reviewer has to find what is wrong with a plausible change and argue for the fix. The skill that matters now is knowing when the AI is wrong, and you cannot outsource that.<\/p>\n

A request can be plausible and wrong as well. I once promoted a DevOps engineer to DevOps lead because, when you gave him a task, he asked questions and pushed back. He tried to understand what you really wanted and how that would develop over the next few years, and built around it.<\/p>\n

Employers and universities are changing the test<\/strong><\/p>\n

Some hiring tests have hit the same wall. Canva tested the questions from its Computer Science Fundamentals interview on AI tools, and the tools \u201cproduced correct, well-documented solutions in seconds,\u201d the company wrote on its engineering blog<\/a> in June 2025. The interview it piloted to replace that screen expects candidates to use AI. One of the things it assesses is: \u201cCan they identify and fix issues in AI-generated code?\u201d GitLab\u2019s careers page<\/a> lets candidates research take-home assignments with AI if they cite it, and asks for answers \u201cin your own words that reflect your original synthesis and perspective.\u201d In technical assessments, it welcomes transparent use of AI \u201cwith the expectation that you can explain your prompts and technical decisions.\u201d<\/p>\n

At the University of Sydney, Adam Bridgeman, pro vice-chancellor for educational innovation, said in November 2024 that \u201cany take-home assessment can be completed to a high level by AI.\u201d The university announced a two-lane approach<\/a> from the second semester of 2025: secure, in-person assessments without external aids, and open assessments that allow AI and other tools. A staff guide<\/a> by Bridgeman and Danny Liu describes open assessments as focused on \u201cacquiring skills and critical thinking processes, rather than grading the outputs of such tasks.\u201d It lists a \u201cQ&A following presentation, submission or placement\u201d among the secure formats proposed for 2025.<\/p>\n

How to set a critique<\/strong><\/p>\n

Start with a plausible piece of work containing flaws you planted. That might be an essay that misreads its main source, or a function that breaks on an empty input. Ask people to find the problems and rank them by how much they matter. Score each answer against the planted flaws and deduct for false alarms. A real problem you missed when you built the task earns credit too. Then have each person defend one finding in conversation while you push back. Sydney\u2019s guide reports that educators using interactive oral assessments find a 15-minute conversation explores student understanding more deeply than reading a written report.<\/p>\n

There is a learning case as well. Bruce McLaren, Deanne Adams, and Richard Mayer studied 390 sixth-grade students<\/a> learning decimals at two Pittsburgh-area schools. One group worked through a fictional student\u2019s solutions, finding and fixing the errors in them. The other group solved the same problems itself. On an immediate test, the two groups did not differ. A week later, the error-finding group did significantly better. They also liked their lesson less.<\/p>\n

The cost moves to whoever sets the task. A planted flaw has to survive a casual read, and the assessor has to understand it well enough to argue about it.<\/p>\n","protected":false},"excerpt":{"rendered":"

TalentAid AG’s Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated. This article originally appeared in Insight Jam, an enterprise IT community that enables human conversation on AI. In the summer of 2023, researchers at the University of Reading in England tested their […]<\/p>\n","protected":false},"author":1511,"featured_media":657,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[14],"tags":[106,108],"acf":[],"yoast_head":"\nGrade the Critique, Not the Artifact: Assessment Once Everyone Generates<\/title>\n<meta name=\"description\" content=\"Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates\" \/>\n<meta property=\"og:description\" content=\"Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/\" \/>\n<meta property=\"og:site_name\" content=\"EdTech Platforms & Learning Management Systems | Solutions Review\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-15T14:28:25+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-15T15:06:26+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Phillip Hamnett\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Phillip Hamnett\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/\",\"url\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/\",\"name\":\"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/edtech\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg\",\"datePublished\":\"2026-10-15T14:28:25+00:00\",\"dateModified\":\"2026-10-15T15:06:26+00:00\",\"author\":{\"@id\":\"https:\/\/solutionsreview.com\/edtech\/#\/schema\/person\/23c30072062eea29dacbe4d43ac96558\"},\"description\":\"Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated.\",\"breadcrumb\":{\"@id\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#primaryimage\",\"url\":\"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg\",\"width\":800,\"height\":400},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/solutionsreview.com\/edtech\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/solutionsreview.com\/edtech\/#website\",\"url\":\"https:\/\/solutionsreview.com\/edtech\/\",\"name\":\"EdTech Platforms & Learning Management Systems | Solutions Review\",\"description\":\"Evaluating Enterprise LMS Software, Virtual Classrooms & Educational Tech Tools.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/solutionsreview.com\/edtech\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/solutionsreview.com\/edtech\/#\/schema\/person\/23c30072062eea29dacbe4d43ac96558\",\"name\":\"Phillip Hamnett\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/edtech\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/Phillip-Hamnett.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/Phillip-Hamnett.jpg\",\"caption\":\"Phillip Hamnett\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/philliphamnett\/\"],\"url\":\"https:\/\/solutionsreview.com\/edtech\/author\/phamnett\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates","description":"Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/","og_locale":"en_US","og_type":"article","og_title":"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates","og_description":"Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated.","og_url":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/","og_site_name":"EdTech Platforms & Learning Management Systems | Solutions Review","article_published_time":"2026-10-15T14:28:25+00:00","article_modified_time":"2026-10-15T15:06:26+00:00","og_image":[{"width":800,"height":400,"url":"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg","type":"image\/jpeg"}],"author":"Phillip Hamnett","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Phillip Hamnett","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/","url":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/","name":"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates","isPartOf":{"@id":"https:\/\/solutionsreview.com\/edtech\/#website"},"primaryImageOfPage":{"@id":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#primaryimage"},"image":{"@id":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#primaryimage"},"thumbnailUrl":"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg","datePublished":"2026-10-15T14:28:25+00:00","dateModified":"2026-10-15T15:06:26+00:00","author":{"@id":"https:\/\/solutionsreview.com\/edtech\/#\/schema\/person\/23c30072062eea29dacbe4d43ac96558"},"description":"Phillip Hamnett offers this commentary on grading the critique, not the artifact and the benefits of assessment once answers are generated.","breadcrumb":{"@id":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#primaryimage","url":"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg","contentUrl":"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/AdobeStock_493222060-1.jpeg","width":800,"height":400},{"@type":"BreadcrumbList","@id":"https:\/\/solutionsreview.com\/edtech\/2026\/10\/15\/grade-the-critique-not-the-artifact-assessment-once-everyone-can-generate-a-decent-answer\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/solutionsreview.com\/edtech\/"},{"@type":"ListItem","position":2,"name":"Grade the Critique, Not the Artifact: Assessment Once Everyone Generates"}]},{"@type":"WebSite","@id":"https:\/\/solutionsreview.com\/edtech\/#website","url":"https:\/\/solutionsreview.com\/edtech\/","name":"EdTech Platforms & Learning Management Systems | Solutions Review","description":"Evaluating Enterprise LMS Software, Virtual Classrooms & Educational Tech Tools.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/solutionsreview.com\/edtech\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/solutionsreview.com\/edtech\/#\/schema\/person\/23c30072062eea29dacbe4d43ac96558","name":"Phillip Hamnett","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/edtech\/#\/schema\/person\/image\/","url":"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/Phillip-Hamnett.jpg","contentUrl":"https:\/\/solutionsreview.com\/edtech\/files\/2026\/10\/Phillip-Hamnett.jpg","caption":"Phillip Hamnett"},"sameAs":["https:\/\/www.linkedin.com\/in\/philliphamnett\/"],"url":"https:\/\/solutionsreview.com\/edtech\/author\/phamnett\/"}]}},"_links":{"self":[{"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/posts\/638"}],"collection":[{"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/users\/1511"}],"replies":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/comments?post=638"}],"version-history":[{"count":0,"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/posts\/638\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/media\/657"}],"wp:attachment":[{"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/media?parent=638"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/categories?post=638"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solutionsreview.com\/edtech\/wp-json\/wp\/v2\/tags?post=638"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}