It’s easy to assume that tech companies are better than regular individuals in distinguishing between content made by AI and manmade content. After all, they specialize in technology, so why wouldn’t they be better? However, a recent experiment shows that these companies must also be more vigilant in preventing people from bypassing their rules with AI.
Interviewing.io, a website helping people succeed in job interviews, tested job interviewers to see if they could detect candidates using ChatGPT in coding rounds. The website team discovered that the AI program can answer their questions so well that it can fool interviewers. As a result, many news outlets like Business Insider reported on the findings.
How did the ChatGPT experiment work?
Business Insider India explains that tech job interviews usually have coding rounds where job applicants solve technical problems. Oftentimes, hiring managers are in charge of these.
They often take questions from the LeetCode interview prep platform. Consequently, candidates can use ChatGPT and other AI programs to cheat.
Interviewing.io proved this phenomenon with an experiment. It had interviewers screen three groups of applicants asking three types of questions:
- Verbatim LeetCode questions
- Modified LeetCode questions
- Custom questions
The website told the interviewers, “[This] research study aims to understand the trends in the predictability of an interviewer’s decisions over time – especially when asking standard vs. non-standard interview questions.”
READ: NASA needs volunteers for Mars simulation
The experiment found that 73% of the verbatim group passed their interviews. More importantly, none of the interviewers caught the ChatGPT cheaters red-handed.
India Today shared a response from Aline Lerner, Interviewing.io’s founder. She said interviewers can get fooled by AI tools because they expect perfect answers from candidates.
READ: A conversation with ChatGPT
“Up until a year ago, it was pretty hard to cheat on those. Now, you can literally just put it in ChatGPT and get an answer in real-time,” Lerner said.
In conclusion, Interviewing.io recommends asking custom questions to prevent cheating with the use of ChatGPT.