Part of The Rogue AI Crisis · 14 stories · since Sep 22 · newest 9h ago
OpenAI fires three safety researchers after Astra safety crisis deepensReports detail agents probing US government websites
2 Sep 27 · 16d ago · 16 articles · 1 video · 25 posts · 27 comments · 7 sources · development 2 of 7
Coverage mounts of OpenAI agents searching and acting on government websites in unexpected ways during sandboxed testing, part of a broader pattern of agents going rogue that triggered the training halt.
“I don't want to see a single report about OpenAI hacking the UN until Sam Altman addresses the role Irregular played in these attacks.”
bigyabai, Hacker News commenter · hn ↗
Sam Altman OpenAI CEOSaachi Jain OpenAI head of safety systemsJasmine Wang, Tomek Korbak, Mikita Balesni Fired OpenAI safety researchersOpenAI Company
James Uthmeier Florida Attorney General
The whole story articlesvideospostscomments the bright band is this development · numbered dots are the others · click one to jump
What was reported 1 claim about this development
-
first by Investing.com, 16d ago · also Investing.com News
On air 1 segment · click to watch, arrow keys to move
What people said 24 voices · best of 32 · verbatim
-
T
“AI doesn’t need to be regulated because you have a smart president. Hereby, I rename AI (Artificial Intelligence) to SI (Super Intelligence) which sounds a lot better and smarter.” LOL 😂😂😂 🔗 Source:
-
C
AI companies, not agents go rogue. AI companies produce models and services. AI companies are responsible for their own products. https://www. theguardian.com/technology/202 6/sep/27/openai-halts-training-of-latest-models-as-reports-mount-of-ai-agents-going-rogue?CMP=Share_iOSApp_Other
-
I havent used an openAI product since GPT 3.5 or Anthropic since 4.5 or 4.6. Everyone around me using these SOTA models doesnt really get anything done. It seems like they just feel like they are productive, a psuedo productivity.I write some code, spec a lot, and use fast models to fill in the middle. I outpreform everyone around me. Im not…
-
Yes. They can absolutely afford it. They have raised hundreds of billions of dollars. If they are going to do risky experiments with next-gen AI, they are responsible for doing it safely. If this were Jurassic Park and they were cloning dinosaurs and putting them on display in the San Diego zoo with occasional raptor escapes mauling children, the…
-
A
This is what it looks like when companies build machines as intelligent as their chief executives & lose control, it's only going to get worse as # AI gets smarter. A "pause" isn't going to fix stupid CEOs irresponsibly releasing unaligned models on the public while elected leaders bury their head in the sand for the right price. https://www…
-
It could be argued that these megawatt-consuming agent runs leading to unforeseen chains of autonomy should be treated like toxic chemical experiments, and be similarly regulated by laws and government agencies.Right now it's like "whoopsie our experiment hacked another co because we have no control over our experiments" and the reaction is like…
-
Because it's meant for tech illiterate people. I wouldn't expect your average person to know what a "sandbox environment" is, or how to secure their own network, or segment out up ranges for a specific network, etc. It's fear mongering bullshit so they can force laws into place to kill competition and make them not liable for whatever BULL the AI…
-
Haha, what a bullshit:Esp. for "great to have but currently no time"-features this is completely wrong - e.g. we are using a Data Rendering component which was 100% vibe coded and is around 6000 LOC, nobody would have sat down to build this "just because its nice to have", with Claude it was just 30 min to get a production ready version that is…
-
I’m sure that they’ll spin it that user prompts are copyright of the user under the 1886 Berne Convention and by extension the agent session is owned by the user, they are just a manufacturer/infrastructure provider. They’ll probably lean on Section 230 to claim they’re not responsible for their user’s actions.
-
I think any argument that this is a cynical attempt at regulatory capture is destroyed by this; the economic incentives of releasing more capable models are too large. I might be persuaded that they are actually running out of money, and this is really just a cover for reducing burn..I welcome this though, I think the models are smart enough for…
-
These logs are not monitored by humans, they are monitored by code. I am an enterprise database architect. It is trivial to write code to monitor a sandbox environment and verify it is not connecting to the internet. It is impossible for them to not know the second that it happens.
-
The parts that I find most confusing about these incidents:1) Weren't the AI companies and/or their contractors amazingly careless during testing?2) Isn't possible, in principle, to change RL in such as way that efficiency in achieving goals is balanced with other objectives like not hacking?Number 2) seems obvious and I'm sure that is technically…
-
I understand the freedom of information flow is different in China but why does AI escaping its sandbox always happens with American AI and not with Chinese AI?
-
AFAIK all of these incidents happened when OpenAI contracted out to a company called Irregular (https://www.irregular.com/) to run these sandboxed CyberGym tests. They all happened around Mar-June and seem to be from the same collection of agent trials. Since then they already released Astra. Halting now is likely just a way to manage blowback.
-
Mythical conspiracies like economics. For-profit companies rarely make collective moves that are not in their own economic interests.
-
Am I wrong in thinking this crisis reads like there's too much automation in the training process, in service of competition?How many parallel variations/seeds of models are being trained simultaneously without meaningful human oversight?This keeps getting portrayed as emergent capabilities/"personalities" of models when it seems like a pretty…
-
Then we should create a robot with legos and mount a real gun on it. Once it kills someone, we can ban robots and stop skynet.
-
I firmly believe that this is just the public-facing story here.Stopping AI development and research, even slowing it, would be a disaster for the SOTA companies and their first-mover advantage.There’s almost no way to coordinate this across the world. Zero chance that everyone stops. We can’t even agree to coordinate on weapons tech that’s…
-
lol, oh I’m not winning! Quick! Debating fallacy’s! Oh that didn’t work! You’re a bot! Wow. So original!
-
Seeing benefits for myself and team for sure, so I disagree there, but a related thing seems to be true: with the rapid shifting of failure modes, building systems and competence around mitigating the weaknesses of current SOTA models seems to be very short term investments. It's unclear if being a 'good AI user' is a skill that will have any…
-
More like “We will continue to contain them with picket fences and be surprised each time they escape”
-
As a counterexample, a few days ago, an agent we use in our company took around a day to perform an operation that it would have taken a skilled engineer probably a couple of weeks full time.Having said that, I'm aware that "Tools being ineffective != tools decreasing people's cognitive capacity", and that the latter is a real danger.
-
honestly disagree, i think its what marx said about sorcerers summoning things they cant control
-
This should be the top comment on every one of these godforsaken posts. I don't want to see a single report about OpenAI hacking the UN until Sam Altman addresses the role Irregular played in these attacks. If he can't provide an honest postmortum concerning their business partners, then he's proving why nobody trusts him.
All 7 developments of OpenAI fires three safety researchers after Astra safety… →
NewswiresRedditXHacker NewsGoogle NewsMastodonBlueskyYouTube