Microsoft sues service for creating illicit content with its AI platform

Microsoft and others are prohibiting the use of their generative artificial intelligence systems to create various content. Prohibited content includes material that depicts or promotes sexual exploitation or violence, is erotic or pornographic, or that attacks, degrades, or excludes people based on race, ethnicity, national origin, gender, gender identity, sexual orientation, religion, age, disability status or similar traits. It is also prohibited to create content that contains threats, intimidation, promotion of physical harm or other offensive behavior.

In addition to explicitly prohibiting such use of its platform, Microsoft has also developed safeguards that examine both the queries entered by users and the resulting output for signs that the requested content violates any of these terms. These code-based restrictions have been circumvented several times in recent years through hacker attacks, some benign and performed by researchers And other malicious threat actors.

Microsoft did not specify how the defendants’ software was allegedly designed to circumvent the company’s barriers.

Masada wrote:

Microsoft AI services employ stringent security measures, including built-in security mitigations at the AI ​​model, platform, and application levels. Microsoft watched as a foreign threat group developed sophisticated software that exploited public customer credentials obtained from public websites, court documents released today allege. In doing so, they sought to identify and illegally gain access to the accounts of certain generative AI services and intentionally change the capabilities of these services. Cybercriminals then used these services and resold access to other attackers with detailed instructions on how to use these special tools to create malicious and illegal content. Following the discovery, Microsoft revoked the cybercriminals’ access, implemented countermeasures, and strengthened security measures to further block such malicious activity in the future.

The lawsuit alleges that the defendants’ service violated the Computer Fraud and Abuse Act, the Digital Millennium Copyright Act, the Lanham Act and the Racketeer Influenced and Corrupt Organizations Act and constitutes wire fraud, access device fraud, violation of common law and harmful interference. The complaint seeks an injunction prohibiting defendants from engaging in “any of the activities described herein.”