Professors Face Challenges with AI Detection Tools in Education

<p>Timothy Paustian, a biology professor at the University of Wisconsin at Madison, has implemented various strategies to prevent students from using AI to write essays. Initially, he tested AI detectors that claimed to identify machine-generated text but found them to be ineffective. He then incorporated hidden prompts into his assignments that only AI would respond to, resulting in the detection of 60 AI-written essays out of 350 submissions in his online microbiology class last year. Most students admitted to using AI, except for one who chose not to appeal the decision to receive a zero on the assignment.</p><p></p><p>Currently, Paustian is reconsidering his approach as students have started to recognize and remove hidden prompts. His university has guidelines advising against sole reliance on AI detectors, prompting him to consider eliminating writing assignments altogether, which he finds disheartening as he believes writing is a valuable learning tool.</p><p></p><p>As AI technology evolves, colleges continue to struggle with the issue of AI-assisted cheating. Chatbots are increasingly capable of completing complex assignments, complicating detection efforts. While AI detection software has improved, there remains significant debate within academia regarding its use. A report from an MIT working group last month cautioned against relying on AI detectors, suggesting they could foster distrust between instructors and students.</p><p></p><p>Professors are caught in a challenging position, with many expressing skepticism towards AI detection tools. Marc Watkins, a lecturer at the University of Mississippi, noted that faculty members feel overwhelmed by the situation. Recently, a tool called Pangram has gained attention as a leading AI detection solution, but Turnitin remains the most widely used tool in higher education. Approximately 1,400 colleges and universities in North America have adopted Turnitin’s AI detection feature, which can automatically analyze student submissions. According to Annie Chechitelli, Turnitin's chief product officer, the tool detected some AI writing in nearly half of all student submissions in the past academic year. However, it has faced criticism for its accuracy, with a reported false positive rate of less than 1 percent and a false negative rate of about 15 percent.</p><p></p><p>Despite the improvements in AI detection, concerns remain about the potential for errors. For instance, Vanderbilt University disabled Turnitin’s AI checker in 2023, citing that even a 1 percent false-positive rate could lead to significant issues if applied to a large volume of student papers. In contrast, Pangram claims a false-positive rate of one in 25,000 based on internal testing, although it has not yet achieved widespread adoption in academic settings.</p><p></p><p>There is a growing consensus among educators that AI should be integrated into learning rather than solely viewed as a tool for cheating. Indiana University’s Kelley School of Business has prohibited the use of AI detectors, encouraging professors to design assignments that promote engagement and critical thinking. Additionally, a Harvard dean recently advocated for a shift away from AI detection towards acceptance of AI in writing-intensive courses.</p><p></p><p>Some educators argue that addressing the challenges posed by AI requires more comprehensive changes to the educational framework. The MIT report emphasized the need for a new “social contract” between teachers and students to effectively navigate the implications of AI in education. Eric Klopfer, director of MIT’s teacher-education program, highlighted the importance of creating assignments that focus on producing valuable outcomes rather than merely assigning grades.</p><p></p><p>While reimagining the educational experience may be a promising response to the challenges posed by AI, it places significant demands on professors, particularly those at institutions with limited resources. Watkins noted that many instructors are simply trying to manage their courses without facing major incidents of academic dishonesty. Paustian indicated that AI has not been a significant issue in his advanced classes, where students engage in original research, but it poses a considerable challenge in his introductory courses with larger enrollment numbers. This situation reflects broader issues in education, where the primary obstacles are often time and resource constraints.</p>