OpenAI shuts down AI detector due to low accuracy

OpenAI quietly discontinued its AI Classifier tool because of its “low rate of accuracy.” Many people, especially teachers, have been demanding a way to detect AI-generated content. Unfortunately, shutting it down may reignite worries artificial intelligence may ruin academic integrity worldwide.

Believe it or not, more people use AI tools to create online content. Still, we need ways to verify the information from these media, especially because more news outlets are trying artificial intelligence. Hence, distinguishing between artificially-generated and humanmade content is more important than ever.

This article will discuss why OpenAI shut down its AI Classifier tool. Then, I will explain why it is difficult to identify AI-generated content, even for AI development firms. Also, I will discuss its impact on daily life.

What was the problem with the AI detector?

Photo Credit: winbuzzer.com

On January 31, 2023, OpenAI released an AI Classifier tool to help users identify AI-made content. Six months later, it discontinued this service with the following update:

“As of July 20, 2023, the AI Classifier is no longer available due to its low rate of accuracy. We are working to incorporate feedback and are currently researching more effective provenance techniques for text.

“And and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated.” OpenAI and Decrypt specified these flaws:

1. It is unreliable for text with less than 1,000 characters. OpenAI says its “reliability typically improves as the length of the input text increases.”
2. The tool may incorrectly flag human-written text as AI-generated. The AI firm said the Classifier “correctly identifies 26% of AI-written text (true positives) as “likely AI-written.” Conversely, it incorrectly flags “human-written text as AI-written 9% of the time (false positives).
3. Also, classifiers that rely on neural networks perform poorly outside their training data. For example, submitting text from Google Bard will likely cause the AI Classifier to make errors.

You may also like: Elon Musk AI pics with Meta CEO goes viral

OpenAI policy research director Lama Ahmad elaborated on the issue with CNN. “We really don’t recommend taking this tool in isolation because we know that it can be wrong and will be wrong at times – much like using AI for any kind of assessment purposes,” she said.

Other people have been offering AI detection services, but they are also unreliable. No AI detection tool can reliably identify AI-generated content at the time of writing.

Why is it so difficult to detect AI-generated content?

Photo Credit: searchenginejournal.com

The main problem with identifying AI-made media is the complexity of AI models. Most AI companies aim to create AI programs that mimic how humans think.

That is why most AI developers often mention the term “neural network” often. Yet, no artificial intelligence program can fully replicate human intelligence.

Modern AI tools rely on large language models, datasets that contain billions of words from different languages. They plot these words on a three-dimensional graph and connect their meanings via algorithms and embeddings.

Ask a question, and the AI program will use these systems to determine the string of words that likely fit your request. It may seem rudimentary compared to a human mind, but that complexity is enough to perplex even its creators.

You may also like: AI-generated drug begins clinical trials

In other words, AI programs are so advanced that programmers can’t fully understand how they provide answers. Believe it or not, Google CEO Sundar Pichai admitted that flaw during an interview with CBS News’ “60 Minutes.”

Pichai stated Google Bard learned Bengali despite never receiving training in that language. Then, the host, Scott Pelley, asked, “You don’t fully understand how it works. And yet, you’ve turned it loose on society?”

“Yeah,” said Pichai. “Let me put it this way. I don’t think we fully understand how a human mind works either,” Pichai stated. This AI tendency is called the “black box problem,” which refers to the phenomenon where a creator does not fully understand how their projects work.

Why must we detect AI-generated content?

Artificial intelligence is supercharging online scams, allowing malicious individuals to produce millions of scam messages in a short amount of time. Also, it harms the intellectual property of various artists.

For example, you can easily find AI-generated songs using the voices of prominent artists. The AI song “Heart on My Sleeve” used the voices of Drake and The Weeknd and became viral.

You may also like: How to boost learning with ChatGPT

More importantly, it risks academic integrity worldwide. Nowadays, many students have been using ChatGPT and similar tools to create homework in seconds.

Some say AI-generated have specific “tells” absent in manmade text. Teachers often claim AI-made text exceeds their student’s capabilities, making it easy to identify.

However, students may require ChatGPT to write at a specific grade level. Also, they can add a few grammatical errors so it looks like they wrote it themselves.

Conclusion

OpenAI shut down its AI Classifier tool because of its low accuracy. It said it is developing ways to detect artificial intelligence content, but it hasn’t released a better one yet.

Educators must adapt to artificial intelligence because it is here to stay. Learn more about these innovative teaching techniques by reading this other article.

This article does not condone using artificial intelligence for academic dishonesty. Follow your school’s policies regarding AI use. Also, check out more digital trends at Inquirer Tech.