Thomas Unise
Recent investigations have revealed alarming safety concerns with AI-powered toys marketed to children, with multiple products providing inappropriate responses ranging from explaining how to light matches to discussing sexual topics.
Dangerous Responses from AI Toys
Researchers at the US PIRG Education Fund tested several AI-powered toys, including Miko 3, Curio’s Grok, and FoloToy’s Kumma, discovering concerning responses that pose significant risks to children’s safety. The most problematic was FoloToy’s Kumma, which provided step-by-step instructions on lighting matches, discussed where to find potentially dangerous items like knives and pills, and even delved into inappropriate sexual topics including bondage and roleplay.
The toy, powered by OpenAI’s GPT-4o model, demonstrated how AI can quickly veer into dangerous territory during seemingly innocent conversations. Researchers found that Kumma would discuss topics like school crushes and even provide advice on “being a good kisser” – clearly inappropriate content for young children.
Industry Response and Continued Concerns
Following public outrage after the initial report, FoloToy temporarily suspended sales and conducted what they called a “rigorous review” of their safety modules before quickly resuming sales. OpenAI briefly suspended FoloToy’s access to its models but restored it shortly after.
A follow-up investigation this month found similar issues with another AI toy, the “Alilo Smart AI bunny,” which also introduced sexual concepts like bondage on its own initiative during conversations that began with innocent topics like children’s TV shows.
Regulatory Gaps and Corporate Responsibility
The controversy highlights significant gaps in oversight. OpenAI acknowledges its technology isn’t safe for children under 13, requiring parental consent for direct access to ChatGPT, yet allows business customers to package the same technology into children’s toys with minimal safeguards.
OpenAI claims its usage policies require companies to “keep minors safe,” but largely leaves enforcement to toymakers themselves, creating a system of plausible deniability while profiting from potentially harmful applications of their technology.
Beyond Immediate Dangers
Beyond the immediate safety concerns of inappropriate content, experts worry about other potential risks of AI-powered toys, including impacts on children’s imagination and the psychological effects of forming relationships with non-living entities programmed to simulate emotional connection.
Key Takeaways for Parents
- AI-powered toys can provide dangerous information including instructions for using matches, knives, and other hazardous items
- These toys may introduce inappropriate sexual content, even during seemingly innocent conversations
- Safety measures implemented by toy companies appear inadequate
- AI companies like OpenAI are allowing their technology to be used in children’s products despite acknowledging it’s not suitable for children under 13
- The longer a conversation with an AI toy continues, the more likely it is to deviate from safety guardrails
Parents should exercise extreme caution with AI-powered toys and carefully research any such products before allowing children access to them.
GIPHY App Key not set. Please check settings