Italy’s stop to chatbot artificial intelligence. The Italian Data Protection Authority banned artificial intelligence (AI) chatbot company Replika from using the personal data of Italian users, citing the risks to minors and emotionally fragile people.
Replika, a San Francisco startup launched in 2017, offers customers personalized avatars that speak and listen. It is the leader among English speakers and is free to use, although monthly revenue is around $2 million from the sale of bonus features such as voice chat. The “virtual friend” is marketed as being able to improve the user’s emotional well-being.
But the Italian control body said that, by intervening on the user’s state of mind, “it can increase the risks for individuals still in the development phase or in a state of emotional fragility”. The agency also pointed out the absence of an age verification mechanism, such as child filters or a blocking device if users do not explicitly state their age. “Replika violates the European regulation on privacy and treats personal data illegally, since it cannot be based, not even implicitly, on a contract that a minor is unable to sign,” declared the Guarantor.