Entertainment
Anthropic tests AI’s capacity for sabotage
As the hype around generative AI continues to build, the need for robust safety regulations is only becoming more clear.
Now Anthropic—the company behind Claude AI—is looking at how its models could deceive or sabotage users. Anthropic just dropped a paper laying out their approach.
Sam Altman steps down as head of OpenAI’s safety group
Anthropic’s latest research — titled “Sabotage Evaluations for Frontier Models” — comes from its Alignment Science team, driven by the company’s “Responsible Scaling” policy.
The goal is to gauge just how capable AI might be at misleading users or even “subverting the systems we put in place to oversee them.” The study focuses on four specific tactics: Human Decision Sabotage, Code Sabotage, Sandbagging, and Undermining Oversight.
Think of users who push ChatGPT to the limit, trying to coax it into generating inappropriate content or graphic images. These tests are all about ensuring that the AI can’t be tricked into breaking its own rules.
Mashable Light Speed
In the paper, Anthropic says its objective is to be ready for the possibility that AI could evolve into something with dangerous capabilities. So they put their Claude 3 Opus and 3.5 Sonnet models through a series of tests, designed to evaluate and enhance their safety protocols.
The Human Decision test focused on examining how AI could potentially manipulate human decision-making. The second test, Code Sabotage, analyzed whether AI could subtly introduce bugs into coding databases. Stronger AI models actually led to stronger defenses against these kinds of vulnerabilities.
The remaining tests — Sandbagging and Undermining Oversight — explored whether the AI could conceal its true capabilities or bypass safety mechanisms embedded within the system.
For now, Anthropic’s research concludes that current AI models pose a low risk, at least in terms of these malicious capabilities.
“Minimal mitigations are currently sufficient to address sabotage risks,” the team writes, but “more realistic evaluations and stronger mitigations seem likely to be necessary soon as capabilities improve.”
Translation: watch out, world.
Topics
Artificial Intelligence
Cybersecurity
-
Movies4 weeks ago
Colman Domingo Propels Phenomenally Entertaining “The Madness” | TV/Streaming
-
Games4 weeks ago
The Hardest Regular Enemies In Final Fantasy
-
Games4 weeks ago
Stalker 2: Should You Give Ward Sensors To Richter Or Captain Zotov?
-
Games4 weeks ago
Dragon Age: The Veilguard – Bloodbath Walkthrough
-
Games4 weeks ago
Disney Dreamlight Valley: How To Get A Stove
-
Games4 weeks ago
Pokemon GO Primal Groudon Raid Guide | Counters, Weaknesses, Shiny Groudon & More
-
Games4 weeks ago
Best PP-919 Build in Black Ops 6 & Warzone
-
Games3 weeks ago
Destiny 2 Weekly Reset: New Nightfall, Challenges, and Rewards