Character.AI, the tech firm behind the globally acclaimed chatbot, announced new teen safety features on Thursday, December 12.
For example, the bot will direct users to the US National Suicide Prevention Lifeline if it detects content referencing self-harm and suicide.
These upgrades come after Character.AI received multiple lawsuits claiming its chatbot is causing teens to harm themselves and others.
What are these new teen safety measures?
The chatbot will lead users to the Suicide Prevention Hotline and implement the following features:
- Parental Controls: These will help parents monitor their time on Character.AI, including time spent on the platform and their favorite Characters.
- Time Spent Notification: The app will tell users when they’ve spent an hour on the platform. Also, users under 18 will have more limits on their ability to modify this feature.
- Prominent Disclaimers: The program will remind users that it’s not a real person, so they must treat its outputs as fiction.
In 2021, former Google engineers Daniel De Freitas and Noam Shazeer founded Character.AI.
The platform lets users create chatbots with unique personalities, also known as “Characters.”
B2B media platform Business of Apps reported that Character.AI had 28 million active users in August 2024, making it one of the most popular chatbots worldwide.
READ: New child safety features for Google and YouTube
The platform can guide people in creating AI chatbots. Eventually, users may develop a passion for artificial intelligence and develop skills highly sought-after in the global AI revolution.
What are some issues with Character.AI?
On the other hand, the program earned infamy as one of the apps letting people create AI girlfriends and boyfriends.
The World Health Organization declared loneliness a “global public health concern,” and some used this technology as a solution.
Filipinos have also turned to similar apps like Replika to alleviate feelings of loneliness.
Read here to learn more about the Philippines’ mental health crisis.
Users can develop parasocial relationships when people emotionally attach to media characters who do not reciprocate their feelings.
In the 1950s, this phenomenon only manifested in fans of pop culture icons like Elvis Presley or Micheal Jackson.
READ: Mother sues AI company, claims chatbot’s interactions led to son’s death
It was also the subject of Eminem’s song, “Stan,” which revolves around a fan who sends letters to his favorite rapper.
Click here to learn more about parasocial relationships.
Nowadays, people can develop stronger parasocial bonds with chatbots like Character.AI as they can respond to users in real-time.
Worse, some interactions allegedly encouraged teens to harm themselves and others.
Last October 30, CNN reported on Florida mother Megan Garcia. She blames Character.AI for the death of her 14-year-old son, Sewell Setzer III.
Consequently, Garcia filed a lawsuit that claimed that his interaction with the bot triggered suicidal ideations.
BBC reported the most recent case on Tuesday, December 10. According to the UK-based news outlet, two families are suing Character.AI after it allegedly “encouraged a teen to kill their parents.”