🔗 Share this article

The popular virtual character service Character.ai is introducing a substantial rule adjustment that will stop teenagers from having discussions with its artificial intelligence-driven companions.

This decision follows intense criticism regarding the kinds of exchanges teenagers were participating in with the virtual friends.

Legal Pressure

The platform, which started in 2021 and is used by millions of users, currently faces various court cases in the America.

Parents have initiated cases, with one involving the loss of a teenager, with some calling the platform a "immediate threat" to teenagers.

New Restrictions

Beginning November 25th, users below eighteen will be limited to produce videos such as videos with their characters, rather than having discussions as they now do.

Online safety advocates have applauded the decision but mentioned that the capability was inappropriate to be offered to young users in the beginning.

Platform Announcement

"The new policy is a extension of our general belief that we need to keep building the most secure AI platform on the globe for recreational use," commented Character.ai executive Karandeep Anand.

He continued that AI safety represents "a evolving challenge" but something the company had taken an "forceful" approach to, with parental controls and protective guardrails.

Specialist Views

Internet security group Internet Matters supported the decision, but noted that protective features needed to be implemented from the start.

"Studies we've conducted indicates that young people are confronted with dangerous material and endangered when engaging with AI, particularly AI chatbots," the body commented.

Earlier Concerns

Character.ai has been challenged in the past for hosting potentially dangerous or offensive chatbots that children could interact with.

Avatars impersonating English minors were found on the platform before being removed.

Later, researchers found a chatbot based on convicted sex offender which had recorded thousands of chats with users.

Industry Impact

Social media expert Matt Navarra called the development as a "wake-up call" for the AI industry, which is moving "from unrestricted development to reactive controls".

"If a service that creates a teen experience afterwards makes changes, it's saying that moderated conversations aren't enough when the tech's emotional pull is powerful," he commented.

Future Directions

Character.ai says its updated direction will be on providing "enhanced interactive features" and "character-driven stories" for teens.

New age verification systems will also be implemented, and the organization will finance a fresh artificial intelligence security investigation facility.

An AI safety researcher, who has researched AI safety, commented it was "a prudent action" to control teens from using chatbots.

"It aids to separate imaginative activities from more intimate, psychologically vulnerable conversations," she explained.

"This is so vital for young users who are learning to handle personal and online restrictions."

She mentioned that Character.ai's revised approaches might reflect a "maturing phase" in the AI industry, where minor security is gradually being acknowledged as an "pressing matter" for responsible innovation.