AI Watermarking Affects Model Responses to Harmful Prompts

<p>In response to a new European Union law, AI platforms are implementing watermarking schemes for the content they generate. Anthropic recently disclosed that its future Claude models will utilize SynthID-Text, an approach developed by Google and released as open source. This method employs a secret key that subtly alters the process a model uses to select the next word in a sentence. For example, a top next word choice might be 'cloudy,' but the key could change it to 'overcast.' Anyone who possesses the key can determine if the content was generated using the platform.</p><p>Recent research indicates that SynthID-Text can not only modify word selection but also influence the tools a model uses and its adherence to safety protocols it has been trained to follow. The risk may increase with adversarial prompts, where an attacker tries to make a model perform harmful actions, such as disclosing passwords or other sensitive information. Instructions that would typically be ignored may, in some instances, be executed when watermarking is applied. This finding highlights the importance for developers to rigorously test how their LLMs and agents behave with watermarking in place.</p><h2>Changing safety behavior</h2><p>“Compared to the same models without watermarking, it is definitely going to change their behavior, especially under adversarial conditions or when these models are called to use tools while powering an agent,” said Andrea Siposova, an AI security researcher at Lasso Security, in an interview with Ars Technica. “Watermarking is designed to be imperceptible to a reader, but we know that any changes to what the model generates will result in tradeoffs, which will manifest in some way.”</p>