A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.
nicce 3 minutes ago [-]
> Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.
Productivity gains are still enormous compared to what we used to do before agents. But, I know that people don't want to stop there.
mixedbit 1 minutes ago [-]
An agent doesn't inherently need wide access to be useful. The most popular application for agents today is writing code. An agent that does this, needs write access to the source code and read/execute access to tools needed to build and test the code, but not much more. There is little added utility from giving such agent access to things like ssh keys.
parsimo2010 3 minutes ago [-]
Agreed- this is the same problem we have with trusted admins or devs who have elevated privileges on their networks. We have to trust that the admins won't use their power to steal company secrets or misuse company resources. If you don't trust the admins, then they can't fix things on your network and there is no point in having them.
If you want an agent to act on its own, like pushing to a git repo, managing dependencies, building and testing, etc., then you have to trust it as much as any other privileged user.
If you don't want to trust it, then you're just forcing yourself into the reverse centaur role, where the agent edits some code, but then has to stop and ask you to push the changes or build the software again and run the unit tests.
Matl 15 seconds ago [-]
> a new chip solves nothing
It does allow Nvidia to sell more chips. This is no genuine attempt to solve anything, imo.
bob1029 11 minutes ago [-]
I feel like we are missing many shades of grey in the middle.
Semi-automation (human in the loop) can still result in a dramatic uplift in productivity. You can't run a combine harvester 100% autonomous but that doesn't stop anyone from trying to get as close to that limit as possible.
inetknght 9 minutes ago [-]
> You can't run a combine harvester 100% autonomous
I'm curious why you think that.
theoreticalmal 5 minutes ago [-]
Probably repair, refuel, what happens in a tornado. There’s an infinite amount of complexity in the world and a finite amount of computation
mschuster91 5 minutes ago [-]
Oh you absolutely can run them autonomously on the field. You only need a human these days to refuel them.
Precision Agriculture stuff is utterly crazy these days, other than fuel the remaining staff is the only thing left where you can get efficiency improvements - and at the scale of modern megafarms, even small percentages add up to a ton of money.
binsquare 3 minutes ago [-]
Running untrusted workloads have been done at scale for a long time.
Every cloud provider dealt with it and concluded that virtual machine technology is an important part of that stack.
Couple it with the right observability, tooling I do think we can curb risks posed by agents.
Legend2440 35 seconds ago [-]
Those workloads have no similarity to agents and are effectively irrelevant.
Either you sandbox it so much that it can't do anything useful; or you allow too much freedom and it can find a way around the restrictions.
The only way out of this dilemma is to find a way to build agents that can be trusted.
johnsmith1840 15 minutes ago [-]
"Inherently needs wide unattended access"
And what if you could? What if you could give a space secure enough it could have direct control over your bank account. It may do something dumb but it's boundaries are beyond the agent.
It could use your routing number and run your gmail without risk of abusing the routing number.
jagraff 7 minutes ago [-]
How would it have access to my routing number and gmail without the risk of sharing my routing number over gmail?
TesterVetter 10 minutes ago [-]
[dead]
luc_ 12 minutes ago [-]
I read this as "let's address our shareholders' concerns with something that will increase shareholder value" mixed with "there's no such thing as 100% secure".
If such hardware were to work... It should almost certainly be open source, and not controlled by a single entity.
Let's watch the stock.
peri-cl 4 minutes ago [-]
It certainly be open source,
> "NVIDIA OpenShell open source software and the NVIDIA Sentry reference system design"
Does this actually do anything other than give a permissions framework for developers who actually want to try to secure their systems?
Do you think the developers at Anthropic, OpenAI and Google who were so sloppy as to not put a good sandbox on their cybersecurity tests before will use this technology correctly? They are supposed to be the experts and they couldn't come up with something similar to this? I am not convinced this voluntary tool will change much of anything.
xg15 16 minutes ago [-]
What does this chip do what a harness with guardrails or running on an account with restricted permissions doesn't do?
chinathrow 13 minutes ago [-]
Generating even more revenue for Nvidia.
lp92 20 minutes ago [-]
So nVidia is trying to sell a new chip to a software and training problem.
downrightmike 4 minutes ago [-]
They are outsourcing their security, rather than in housing it, throw another chip in there.
It was the same firm, Irregular's harness that pushed all of the AI's to break free and hack. Given they did it with each AI, that's exactly what they wanted to do
lambdaone 23 minutes ago [-]
The Sentry chip has to be get it right every time; the contained ASI only has to be lucky once.
brcmthrowaway 6 minutes ago [-]
The bomber always gets through?
toasty228 21 minutes ago [-]
Quis custodiet ipsos custodes?
asdf88990 8 minutes ago [-]
It is Custodians all the way son, you can’t fool me!
Kuyawa 2 minutes ago [-]
China please save us!
Come take all our liberties, our money, our newborns, our fingers so we can't code anymore, but please save us from this madness!
ErrantX 9 minutes ago [-]
I do think that Taylor's 2025 "Not Till We Are lost" should be required reading for anyone deeply involved in AI, Agents, etc.
It was prescient (especially given he'd have written it through 2024) in its depiction of the ability of an AGI to break its boundaries.
Ultimately the risk of AI breakout(s) come down to the weakest human link.
joshstrange 17 minutes ago [-]
Chipmaker thinks the answer is more chips... No surprise.
At the current state of LLM-tech I'm completely opposed to any kind of "watchdog" concept just like I'm opposed to banning open models, regulatory capture, etc.
I'd rather we all have access to these tools then to keep them sequestered by the largest/most-powerful governments (which is the natural outcome for any of this "slow down" bullshit).
dopplr 11 minutes ago [-]
Just hold AI labs blanket liable for ALL harms caused by AI. Actually charge the two labs (so far) with criminal violations of the CFAA and hold them accountable. That is truly the only way these companies will be more careful as a whole, and while I am certain the lawyers of these lab disagree, I think there is some appetite from dario, musk, and sam for broad and strong regulation so that everyone has to slow down instead of just one lab doing it voluntarily and everyone else scurrying past them
of course they do. the more silicon they can sell, the more profit they produce.
philipwhiuk 27 minutes ago [-]
It's amazing that the solution devised by a chip manufacturer to a problem is selling another chip.
cartersj 24 minutes ago [-]
This feels suspiciously good for Nivida, yes.
I wonder how this will impact other chip manufacturers? What about people running local models on older hardware? Does this imply vendor lockout is coming in the future or is this restricted to datacenter hardware?
chinathrow 24 minutes ago [-]
TPM all over the place, again.
happyPersonR 11 minutes ago [-]
lol time to buy some fpga’s … even if they’re slow
sehw 13 minutes ago [-]
[dead]
dist-epoch 27 minutes ago [-]
HN'ers which complained that "OpenAI can't design a proper sandbox, it's so easy, why wouldn't you airgap the network"? will now be "this is outrageous, more software lock-in, walled garden, war against general compute, next year they will put it in your laptop"
ssl-3 3 minutes ago [-]
That a person can see such endless pages of people having various forms of disagreement, and yet somehow manage to conclude that this observed chaos constitutes a clear exhibition of cohesive groupthink is just...stunningly amazing to me.
I don't know why I find it so amazing since it happens with such regularity, but I'm always amazed by it anyway.
HPsquared 27 minutes ago [-]
Both can be true at the same time.
johnsmith1840 12 minutes ago [-]
Airgap what network? How is it gonna order you a burrito on doordash without a network?
Or push to github?
Dylan16807 8 minutes ago [-]
That's for when they're doing hacking tests that aren't supposed to be connected to the internet.
wyre 1 minutes ago [-]
My question with this point is that OpenAI’s office (or any office doing agentic research, really) is not in the same building as the DC that powers the models, so isn’t the only way to access the models over the internet?
mattmcal 24 minutes ago [-]
This is like using "protect the children" as an argument for dragnet surveillance.
You're only revealing your own inability to appreciate the nuance between these two situations.
applfanboysbgon 17 minutes ago [-]
Where is the contradiction? There is a trivial solution that does not impinge on our freedoms, so why on Earth would the existence of the trivial solution that could be used to avoid the tyrannical solution justify accepting the tyrannical solution?
speedgoose 24 minutes ago [-]
So?
Rendered at 16:29:58 GMT+0000 (Coordinated Universal Time) with Vercel.
Productivity gains are still enormous compared to what we used to do before agents. But, I know that people don't want to stop there.
If you want an agent to act on its own, like pushing to a git repo, managing dependencies, building and testing, etc., then you have to trust it as much as any other privileged user.
If you don't want to trust it, then you're just forcing yourself into the reverse centaur role, where the agent edits some code, but then has to stop and ask you to push the changes or build the software again and run the unit tests.
It does allow Nvidia to sell more chips. This is no genuine attempt to solve anything, imo.
Semi-automation (human in the loop) can still result in a dramatic uplift in productivity. You can't run a combine harvester 100% autonomous but that doesn't stop anyone from trying to get as close to that limit as possible.
I'm curious why you think that.
Precision Agriculture stuff is utterly crazy these days, other than fuel the remaining staff is the only thing left where you can get efficiency improvements - and at the scale of modern megafarms, even small percentages add up to a ton of money.
Every cloud provider dealt with it and concluded that virtual machine technology is an important part of that stack.
Couple it with the right observability, tooling I do think we can curb risks posed by agents.
Either you sandbox it so much that it can't do anything useful; or you allow too much freedom and it can find a way around the restrictions.
The only way out of this dilemma is to find a way to build agents that can be trusted.
And what if you could? What if you could give a space secure enough it could have direct control over your bank account. It may do something dumb but it's boundaries are beyond the agent.
It could use your routing number and run your gmail without risk of abusing the routing number.
If such hardware were to work... It should almost certainly be open source, and not controlled by a single entity.
Let's watch the stock.
> "NVIDIA OpenShell open source software and the NVIDIA Sentry reference system design"
https://nvidianews.nvidia.com/news/open-agent-safety-platfor...
Do you think the developers at Anthropic, OpenAI and Google who were so sloppy as to not put a good sandbox on their cybersecurity tests before will use this technology correctly? They are supposed to be the experts and they couldn't come up with something similar to this? I am not convinced this voluntary tool will change much of anything.
It was the same firm, Irregular's harness that pushed all of the AI's to break free and hack. Given they did it with each AI, that's exactly what they wanted to do
Come take all our liberties, our money, our newborns, our fingers so we can't code anymore, but please save us from this madness!
It was prescient (especially given he'd have written it through 2024) in its depiction of the ability of an AGI to break its boundaries.
Ultimately the risk of AI breakout(s) come down to the weakest human link.
At the current state of LLM-tech I'm completely opposed to any kind of "watchdog" concept just like I'm opposed to banning open models, regulatory capture, etc.
I'd rather we all have access to these tools then to keep them sequestered by the largest/most-powerful governments (which is the natural outcome for any of this "slow down" bullshit).
I wonder how this will impact other chip manufacturers? What about people running local models on older hardware? Does this imply vendor lockout is coming in the future or is this restricted to datacenter hardware?
I don't know why I find it so amazing since it happens with such regularity, but I'm always amazed by it anyway.
Or push to github?