Itdaily - OpenAI introduces text watermarking for EU to comply with AI Act

OpenAI introduces text watermarking for EU to comply with AI Act

Phone screen showing the OpenAI logo next to the colorful Codex logo on a black background.

OpenAI is rolling out text watermarking in the European Union to comply with the EU AI Act. The rollout begins with eligible ChatGPT and Codex subscriptions in the European Union.

Since the new transparency rules of the AI Act came into effect on August 2, providers of generative AI are required to mark AI-generated content as such and make it machine-readable. OpenAI is starting with invisible watermarks on texts generated by ChatGPT and Codex in the European Union to comply with the EU AI Act.

“Starting today, API customers worldwide can opt-in to text watermarking for selected models. Text watermarking remains disabled by default in the API,” OpenAI stated in a blog post. The watermarks will be rolled out over the coming weeks to eligible ChatGPT and Codex users on all plans in the EU.

TextGrain watermarks

OpenAI’s watermarking technology, textGrain, adds an invisible statistical signal to the model’s choice of words. The detector analyzes passages for that signal to determine if an OpenAI watermark is present. According to OpenAI’s technical report, textGrain performs at least as well as other tested methods such as SynthID for text.

Under ideal conditions, the tool detects a large portion of the watermarked text, but OpenAI emphasizes that this may be less reliable in daily use. The technology will eventually be made available as open source so that other parties can research and build upon it. For organizations, this should make it more transparent which pieces of text have been influenced by OpenAI models.

Editing weakens the watermark

The detection rate depends heavily on text length and type. At a desired false positive rate of one percent, the detector recognized approximately 80 percent of watermarked passages with 200 tokens and about 95 percent at 400 tokens in psychology content. For texts with little variation in word choice, such as mathematics, detection rates are significantly lower.

Chart: watermark detection drops sharply when words are replaced, to 17% at 25% replaced words
The impact of edits on detection. Source: OpenAI

Editing text weakens the watermark quickly. In passages of 400 tokens, detection dropped from about 92 to 66 percent when ten percent of the words were replaced by synonyms. With 25 percent replacement, that dropped further to seventeen percent. “These limitations contribute to our decision to provide initial detector access only to approved researchers and expert organizations, who can help us evaluate reliability and responsible use,” the company said.

No proof of ‘human authorship’

Furthermore, OpenAI warns that a missing watermark “does not prove human authorship.” “Watermarks can indicate that an OpenAI system generated or processed part of a passage, but not how much human judgment, editing, or creativity went into it,” the company stated.

The new transparency rules are setting a larger movement in motion for AI providers. In addition to OpenAI, Anthropic, Google, and Microsoft, among others, agreed to the labeling requirement in the EU. In August, Anthropic started watermarking AI-generated text and images, as did Microsoft in Copilot and Google in Gemini.