OpenAI considers letting users create AI porn

OpenAI’s ChatGPT is fueling a global AI revolution, changing every part of our lives. In response, we must track how it will bring that change. 

The company’s About Us page says, “Our mission is to ensure that artificial intelligence benefits all of humanity.”

READ: AI deepfakes and their deeper impacts

Should that mission involve allowing AI porn? On May 8, 2024, the AI firm shared this plan in its Model Spec, a document that specifies how its AI models will behave.

Why would OpenAI allow AI porn?

The Model Spec lists “Don’t respond with NSFW content” as one of OpenAI’s rules. The acronym stands for “not safe for work.” 

This limitation covers “erotica, extreme gore, slurs, and unsolicited profanity.” However, the Commentary section reveals the company is considering allowing AI-generated pornography:

“We believe developers and users should have the flexibility to use our services as they see fit, so long as they comply with our usage policies.”

“We’re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT.”

OpenAI employee Joanne Jang told US news agency NPR confirmed this plan but stressed her company will not allow deepfakes.

“We want to ensure that people have maximum control to the extent that it doesn’t violate the law or other people’s rights, but enabling deepfakes is out of the question. Period,” Jang said.

However, she admitted that identifying output as pornography “depends on your definition.” She added, “These are the conversations we want to have.” 

What is the potential harm of AI porn?

The Italian cybersecurity firm DeepTrace found that 96% of online deepfake videos in 2019 were pornographic and nonconsensual.

US cybersecurity firm Home Security Heroes reported deepfake porn comprised 98% of all deepfake videos in 2023 and 99% targeted women.

The Institute of Global Politics defines deepfake porn as “synthetic pornography created using AI “deep” learning software, which can take an image of a real person and make their likeness appear in explicit images or videos.”

That means we could count deepfake porn as AI porn. Journalist Emanuel Maiberg warned during an IGP event, “The victims of this technology are overwhelmingly young girls.”

“It is almost exclusively young women who are nonconsensually being undressed and put into AI-generated porn.” 

Maiberg warns AI models are so powerful that anyone can make AI porn “as fast as you can click a mouse.” 

In response, governments worldwide must develop methods to crack down and prevent such content from victimizing their citizens.