OpenAI, the creators of ChatGPT, have clearly heard the critiques about its AI chatbot. Namely, those from educators who have voiced concern regarding students using ChatGPT to do their homework or write their papers for them.
On Tuesday, OpenAI released a new AI classifier tool aimed at directly addressing those concerns. The free web tool was created by OpenAI in order to help users determine whether a block of text was written by a human or a computer.
The tool is simple to use. Simply visit the AI classifier, log in, and copy and paste the block of text you want to check. The classifier will then rank the text on a scale as either likely to be AI-generated, possibly, unclear, unlikely, or very unlikely.
Credit: OpenAI / Screengrab
While this sounds incredibly helpful in tackling some concerns about AI written text, OpenAI does make it very clear that there are caveats. For example, according to OpenAI, “the classifier is not fully reliable.”
“In our evaluations on a ‘challenge set’ of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as ‘likely AI-written,’ while incorrectly labeling human-written text as AI-written 9% of the time (false positives),” said the company in a blog post.
There are other limitations with the classifier addressed by OpenAI in its announcement. For example, the classifier works considerably worse with text that is less than 1,000 words. Sometimes it can falsely identify human-written content as text created by a computer. And it only really works well with English-language text.
OpenAI isn’t the first to release a tool to help detect when text is computer-generated. In fact, its not the first classifier tool the company itself has released. However, it’s “significantly more reliable” than its previous tool, according to OpenAI. And its very notable that such a tool is continuously being worked on by one of the leaders in the AI space as well.
Still, there’s quite a way to go before teachers and educators can confidently rely on any AI-detection tool to suss out cheating in the classroom.
