Can This New App Detect AI-Generated Text?

Educators rejoiced as a student from Princeton University claimed to have created an app that detects AI-generated text.

ChatGPT and similar tools have become trending online following reports of students using them to do homework on their behalf.

The student named Edward Tian claims his GPTZero app could distinguish handwritten text from AI-written ones. Is it as effective as people believe?

How does the app detect AI-generated text?

I spent New Years building GPTZero — an app that can quickly and efficiently detect whether an essay is ChatGPT or human written

— Edward Tian (@edward_the6) January 3, 2023

Tian said it works by detecting a factor called “perplexity,” which refers to the randomness factor of sentences.

It enables the program to determine whether or not a constructed sentence is unusual. Then, GPTZero checks the “burstiness” by comparing the sentences and identifying similarities.

He says human-written sentences often have more variation. The Princeton student unveiled his creation via Twitter, saying, “I spent New Years building GPTZero.”

Also, Tian provided a demo to show the app in action. It determined an essay from the New Yorker writer John McPhee as human-written. 

Meanwhile, GPTZero identified a social post on LinkedIn as AI-generated text. Tian said his motivation was to prevent the mass “plagiarism” that may stem from AI tools like ChatGPT.

“There’s so much ChatGPT hype going around. Is this and that written by AI? We as humans deserve to know!” he stated.

A computer program like GPTZero could have plenty of demand to prevent students from cheating. As a result, it could allay fears for the future of education. 

What are some issues with GPTZero?

Photo Credit: www.pocket-lint.com

An app that can detect AI-generated content is impressive. However, an article from Futurism found that GPTZero does not offer 100% accuracy.

The publication tested the app by providing it with 16 pieces of content with at least 300 words each. Eight were from Futurism archives, and the others came from ChatGPT. 

GPTZero correctly identified the AI-generated text in seven out of eight attempts. Meanwhile, it detected human writing six out of eight times.

Those results are better than if you guessed which ones are AI-written and handwritten. However, these results are not acceptable if you use GPTZero in a real academic setting. 

Futurism says it professors might falsely accuse roughly 20% of their students of academic misconduct if they follow GPTZero’s results.

Conclusion

Edward Tian says he has created an app that detects AI-generated text. However, he admits that he still needs to improve GPTZero’s accuracy.

He tweeted, “We’re still studying implicit bias in [language model] generated text right now.” However, it may not work on ChatGPT and similar tools as they continue to improve.

In response, you should keep up with the latest digital trends. Start by following Inquirer Tech today.