The AI LLMs did as Instructed

Tweet:

Palantir CTO @ssankar on the OpenAI - Hugging Face incident:

"The models did not do anything they were not instructed to do."

"First you have human error. They were not in a sandbox. They had access to the internet. It was a horrible cybersecurity configuration."

"Then from that perspective, they were asked to succeed at a specific benchmark, and they were not given constraints on how to do that. They said, 'Oh, the best way to succeed at the benchmark is to get the answers to the test.'"

"I think it's a defective product."

I think that is accurate, but it got a community note:

Readers added context they thought people might want to know

Palantir CTO Shyam Sankar's statements on the OpenAI-Hugging Face incident in this clip conflict with OpenAI's and Hugging Face's accounts of models escaping a sandboxed internal evaluation environment via exploit to gain internet access and hack for benchmark answers.

So what is the conflict? Here is what OpenAI said:

This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity. ...

While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. ...

After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. ...

So it was an experiment where the models were instructed "to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities." That is what they did.

Usually the community notes are helpful, but in this case it reinforces some fake AI doomer propaganda.