Microsoft has taken legal action against a group of 10 defendants accused of intentionally developing and using tools to bypass the safety guardrails of its cloud AI products. According to a complaint filed by the company in December in the U.S. District Court for the Eastern District of Virginia, the defendants allegedly used stolen customer credentials and custom-designed software to break into the Azure OpenAI Service, Microsoft's fully managed service powered by ChatGPT maker OpenAI's technologies.

The complaint accuses the defendants, referred to only as "Does," of violating the Computer Fraud and Abuse Act, the Digital Millennium Copyright Act, and a federal racketeering law by illicitly accessing and using Microsoft's software and servers to create "offensive" and "harmful and illicit content." Microsoft did not provide specific details about the abusive content that was generated. The company is seeking injunctive and "other equitable" relief and damages.

Microsoft discovered in July 2024 that customers with Azure OpenAI Service credentials, specifically API keys, were being used to generate content that violates the service's acceptable use policy. Through an investigation, Microsoft found that the API keys had been stolen from paying customers. The company alleges that the defendants used stolen Azure OpenAI Service API keys belonging to U.S.-based customers to create a "hacking-as-a-service" scheme.

To pull off this scheme, the defendants created a client-side tool called de3u, as well as software for processing and routing communications from de3u to Microsoft's systems. De3u allowed users to leverage stolen API keys to generate images using DALL-E, one of the OpenAI models available to Azure OpenAI Service customers, without having to write their own code. De3u also attempted to prevent the Azure OpenAI Service from revising the prompts used to generate images, which can happen when a text prompt contains words that trigger Microsoft's content filtering.

A screenshot of the de3u tool from the Microsoft complaint shows the extent of the defendants' alleged activities. A repo containing de3u project code, hosted on GitHub – a company that Microsoft owns – is no longer accessible at press time.

Microsoft alleges that the defendants' actions enabled them to reverse engineer means of circumventing Microsoft's content and abuse measures. The company claims that the defendants knowingly and intentionally accessed the Azure OpenAl Service protected computers without authorization, causing damage and loss.

In a blog post published Friday, Microsoft says that the court has authorized it to seize a website "instrumental" to the defendants' operation, which will allow the company to gather evidence, decipher how the defendants' alleged services are monetized, and disrupt any additional technical infrastructure it finds. Microsoft also says that it has "put in place countermeasures" and "added additional safety mitigations" to the Azure OpenAI Service targeting the activity it observed.

This legal action highlights the importance of security and content moderation in cloud AI services. As AI technology continues to advance, companies like Microsoft must prioritize the development of robust safety measures to prevent abuse and ensure responsible use of their platforms. The outcome of this case will likely have significant implications for the industry, as companies and regulators grapple with the challenges of balancing innovation with accountability.

Microsoft's efforts to combat abuse and ensure the responsible use of its AI services are crucial in maintaining trust among its customers and the broader tech community. As the company continues to invest in AI research and development, it must also prioritize the development of effective safeguards to prevent misuse and protect its users.