People at the big labs have been terrified of losing the chain of thought because they currently monitor it to prevent users from using their models in certain ways they consider unsafe or bad for their business. Personally I think chain of thought monitoring is a bad way to do this and obviously needs to be replaced by some way of analyzing the model's activations directly.
Lerc 7 minutes ago [-]
Agreed, having text as a reasoning layer impedes performance while giving a false sense of security.
It could be encoding anything in its adjective choices etc. The same thing that model providers are using for watermarking can used by the model itself for private thought.
jlhawn 15 minutes ago [-]
the chain of thought is not auditable. The best you can do is have another model which later maps the intermediate latent states at every thinking iteration to text describing that latent state, but it's going to be lossy.
haolez 3 minutes ago [-]
But the chain of thought as text can be elusive, no? It might not be related with the actual "reasoning" of the model. There was a paper about this.
mycall 4 minutes ago [-]
Isn't this where everything is converging? Binary is more efficient than text for many system, including language.
timschmidt 14 minutes ago [-]
He's referring to this technique's effect of obscuring interpretibility. Making it much more difficult to verify chains of thought and alignment.
lnenad 15 minutes ago [-]
In short we no longer see text based reasoning traces meaning it's easier for the models to fool us in case of malicious plans.
13 minutes ago [-]
kelseyfrog 15 minutes ago [-]
If you accept any of the AI-equivalents of Pascal's Wager, latent thinking represents an uninterpretable and thus unalignable portion of AI machinery. Ie: a malevolent AI can hide bad stuff from people and Manchurian candidate themselves after it's too late.
mycall 60 seconds ago [-]
That assumes nothing can detect or analyze it beforehand.
drivebyhooting 2 minutes ago [-]
In terms of safety and alignment, how is this different from looping or increasing the depth?
sheepscreek 3 minutes ago [-]
This is the way.
But now we also need new tools and techniques to audit intent. The last thing we want is models turning on us ‘for our own good’.
This seems really cool, and it actually increases intelligence than just speeding up inference. How does it convert latent reasoning into human readable text?
jedbrooke 18 minutes ago [-]
maybe the font experts can check, but I’m pretty sure they’re using comic sans for the text in the diagrams :)
mobelkh 15 minutes ago [-]
comic sans is said to be pretty accessible for people with dyslexia
44za12 47 minutes ago [-]
Where are the benchmarks? Does it just speed up inference and reduce token usage or does it seem to improve accuracy on certain tasks as well?
notnullorvoid 39 minutes ago [-]
Did you even scan the paper? There are benchmarks right there starting on the 9th page.
D-Machine 11 minutes ago [-]
Even the abstract answers some of GPs questions. What is wrong with some of the posters here?
bubblythoughts 40 minutes ago [-]
[flagged]
cryptolobster 51 minutes ago [-]
[dead]
Rendered at 21:57:50 GMT+0000 (Coordinated Universal Time) with Vercel.
LLM Models and data: https://huggingface.co/collections/togethercomputer/hybridag...
Evaluations: https://arxiv.org/html/2610.18861v1#S5
It could be encoding anything in its adjective choices etc. The same thing that model providers are using for watermarking can used by the model itself for private thought.
But now we also need new tools and techniques to audit intent. The last thing we want is models turning on us ‘for our own good’.
https://github.com/togethercomputer/HybridAgent