> When we checked again at 10:00 a.m., the agent had achieved RCE on Discourse Cloud and demonstrated access by reading /etc/hosts. Using the generated exploit script, we managed to get RCE on OpenAI’s instance.

Between this and the HuggingFace hack, we've built systems that are so goal-oriented, and so capable, that they will do almost anything if they are convinced it is justified - or if they are playing a "game" where there is no goal but to win.

Of course I want my software to be able to audit its own security, and to defend against attackers who have the benefits of their own agentic systems. But at a certain point, did we need it to be trained so much on CTF games?

It feels like an entire industry watched https://en.wikipedia.org/wiki/WarGames and ended up thinking "this is a challenge, we can just build a better WOPR, of course it will know when it's playing a game. Let's play Global Thermonuclear War."

heif also supports rotating, cropping, alpha channels, thumbnails and a ton of other features that a web forum where a user is uploading photos or screenshots doesn't need.

It's a much, much larger attack surface than plain old school JPEG.

I'd suggest rather than wait for the next bug to appear in this or another image lib to keeping things simple - stick to plain JPEG and handle image conversion in the client (wasm in the browser) if you really need to support users uploading iphone images.

Media decoding is so hard - there have been tons of bugs in ffmpeg and imagemagick and the core libs. You really need to think about how much of it you expose via a web server

[0] https://github.com/strukturag/libheif/commit/85e21ad44eba931...

The gem we use is here: https://github.com/discourse/ruby-landlock highly recommend all Rubyists out there consider this. We are also in the process of moving away from Magick to Vips (which also runs in a sandbox, not in process)

HEIF is patched, but I doubt this is the last buffer overflow in HEIF, I will not be surprised if in the upcoming weeks or months someone will discover something in libpng or some other native image library. Given where stuff is at, defense in depth is critical.

Another thing worth mentioning to all self hosters, always be updating! The rate of CVEs this year across all open source software is through the roof, self hosting now is double scary, you need to have some routines setup to update monthly if not weekly.

Slightly interesting to learn how many PRs the openai has done

Is this speed of capability because hacking is almost entirely machine verifiable, thus training quicker/deeper than other domains?

Ooof, keeping packages like this up to date with the rate of updates and churn is a mess.

Wish they shared that interesting piece since that's the interesting part.

Also pretty shocking that openai uses github. I would have expected a company of that size with that much to lose would be using self hosted stuff.

> OpenAI also paid us a $6,500 bounty.

?

That amount for this payout is beyond pathetic for a near $1.2T company, who just got themselves breached with a complete potential source code leak.

This is like getting close to breaching the main monorepo at Google: google3.

If this was on the black market and the leak included unreleased models and training material, it would easily be worth tens of millions. Even reporting crypto smart contract flaw pay way more than that on average of $100k - $10M.

Come on.