What would possibly this coordinated effort really obtain? Contemplate the Hugging Face assault once more. OpenAI has stated that the mannequin that drove a lot of the rogue brokers was a “extremely persistent” next-generation mannequin that it was testing in-house. Their implication seems to be that OpenAI has constructed a mannequin so good it’s harmful.
However in the event you learn the reviews in regards to the Hugging Face hack printed by OpenAI and METR, a third-party agency that OpenAI referred to as in to assist them perceive what occurred, what you come away with is the impression not of a mannequin that was too highly effective for OpenAI to maintain up with, however of a damaged mannequin that OpenAI failed to coach correctly.
The brokers did what they did—together with leaving messages for each other, delegating work to different brokers, and scouring their setting for any means attainable to finish their duties—as a result of they’d been rewarded throughout coaching for doing precisely these issues. There have been additionally errors within the coaching setup, reminiscent of duties that had been not possible to finish, which pushed the fashions to search out surprising workarounds that had been additionally rewarded. On the time, many of those points went missed or unreported.
OpenAI says it has stopped coaching this new mannequin and locked it down. That makes it sound prefer it has caged a harmful beast. The truth is, OpenAI has shelved a defective product.
That’s to not say a defective product can’t be harmful. Damaged software program has even killed folks prior to now. However because the dialogue of a slowdown gathers steam, it’s value remembering that every one of that is self-inflicted. A slowdown may need some altruistic unwanted side effects. Nevertheless it’ll principally give these tech titans an opportunity to wash up the mess on their very own meeting traces.
Transparency from these frontier labs can be key to any significant effort to reform, restrain, or regulate AI. In any other case, the remainder of us will nonetheless solely have their phrase for precisely what they’ve constructed and the way secure it’s—no matter tempo they’re going.
To proceed this dialogue about AI’s newest doomer second, be a part of me and my colleagues for a subscriber-exclusive Roundtable dialogue tomorrow, September 15, at 11 a.m. US jap time. We hope to see you there!
