Colleges Address Challenges of AI Use in Student Writing

<p>Timothy Paustian, a biology professor at the University of Wisconsin at Madison, has implemented various strategies to prevent his students from submitting essays generated by AI. Initially, he tested AI detectors that were ineffective. He then incorporated hidden prompts into assignments that only AI would respond to, resulting in the identification of 60 AI-written essays out of 350 submissions in his online microbiology class last year. Most students admitted to using AI, except for one who chose not to appeal the zero grade given for the assignment.</p><p></p><p>However, Paustian is now reconsidering his approach as students have started to recognize and remove the hidden prompts. Additionally, his university has guidelines that discourage reliance on AI detectors. Consequently, he plans to eliminate writing assignments this year, expressing disappointment as he believes writing is a valuable learning tool.</p><p></p><p>As the use of ChatGPT continues to evolve, colleges are struggling to address the issue of AI cheating. Chatbots are increasingly capable of handling complex assignments, making detection more challenging for educators. Although AI detection software has improved, there remains a significant debate within academia regarding its use. An MIT working group recently released a report advising against reliance on AI detectors, citing concerns about fostering distrust between instructors and students.</p><p></p><p>Many professors feel overwhelmed by the situation. Marc Watkins, a lecturer at the University of Mississippi, noted the confusion surrounding AI's impact on education. A tool called Pangram has gained recognition for its AI detection capabilities, while Turnitin remains the most widely used tool in higher education, with around 1,400 institutions in North America utilizing its AI detection feature. Turnitin claims its detector has a false positive rate of less than 1% but a false negative rate of about 15%, meaning it may misidentify AI writing as human work.</p><p></p><p>Despite the low rates, institutions like Vanderbilt University have disabled Turnitin's AI checker due to concerns about false positives. The university noted that even a 1% false positive rate could lead to significant issues if they process around 75,000 student papers annually.</p><p></p><p>Pangram claims a false positive rate of one in 25,000 based on internal testing. A working paper from the University of Chicago indicated that Pangram rarely mistakes long human-written passages for AI-generated text. However, it may err on shorter snippets. Both Pangram and Turnitin assert that their tools do not exhibit bias against non-native English speakers, despite some studies suggesting otherwise.</p><p></p><p>There is a consensus among educators that AI detectors should not be the sole basis for determining academic integrity. Turnitin's Chechitelli emphasized the importance of using detection tools as part of a broader conversation between teachers and students. However, as detection accuracy improves, there is a risk that professors may rely too heavily on these tools, potentially leading to false accusations of academic misconduct.</p><p></p><p>Recent research from Notre Dame highlighted another challenge: even accurate AI detectors can be deceived by tools that modify writing to evade detection. This creates a complex dynamic where students using AI-generated content may have an advantage over those who write their own essays but use editing tools.</p><p></p><p>Some educators advocate for the integration of AI into the learning process rather than focusing solely on detection. Indiana University's Kelley School of Business has prohibited the use of AI detectors, encouraging professors to design assignments that promote engagement and critical thinking. Similarly, a Harvard dean recently suggested moving away from AI detection in favor of embracing AI in writing-intensive courses.</p><p></p><p>Other educators argue that a more comprehensive approach is necessary to address the implications of AI in education. MIT's AI report emphasized the need for a new social contract between educators and students, advocating for assignments that prioritize meaningful learning experiences over traditional grading metrics.</p><p></p><p>While reimagining educational practices may be an ideal response to the challenges posed by AI, it places additional demands on professors, particularly those at institutions with limited resources. Watkins noted that many instructors are focused on managing their courses without encountering significant cheating incidents. Paustian observed that AI has not been a major issue in his advanced classes, where students engage in original research, but it poses significant challenges in his introductory course with larger enrollments. This situation reflects broader issues in education, where the primary obstacles are often time and resource constraints.</p>