I have not looked carefully but it seems like this is over-promising on avoiding catastrophic forgetting.
The "trunk learning rate" is set at 0.1x the learning rate for the experts, so learning on different subjects disproportionately happens in the experts, and the trunk portion is comparatively more stable. But the population of experts can grow and shrink:
> The pool grows when it is short of capacity and shrinks when parts of it stop being asked for.
So:
- doesn't the trunk then _eventually_ still undergo catastrophic forgetting, it just may take much longer?
- and before that point, catastrophic forgetting happens in stepwise chunks whenever the expert pool shrinks?
jmatthews 2 hours ago [-]
This is an interesting approach. First of all, thanks for sharing your work. I've done. I want to say similar work in that I have trained continuous learning models and I have also offloaded parametric knowledge to hard drive people underestimate how difficult that is to do in a functional model. I look forward to digging in deeper.
whizzter 9 hours ago [-]
Nobody will throw rocks, I think most people are curious/suspicious about the big players and wants more hands-on since we suspect that this all will come down in cost soon enough.
ilaksh 4 hours ago [-]
If you actually scroll through the transcript he links to, you will see that something that looks like it could be training is happening, but no coherent responses are coming out at any point. At least not that I saw skimming through.
That might explain why there are no benchmarks of any kind.
synctext 4 hours ago [-]
Using the term AGI and not including any performance analysis. My AI calls it: "massive marketing overreach".
Somebody called this slop in the comments.
As a professor who published on continual learning I'm leaning towards agreement[1]. It lacks any substance. No relation to related work, no description of algorithm, no ablation study, just hand-waving that we're feeding some data and "Chess is not forgotten".
This "how-continual-learning-works" markdown text is not an algorithm [2].
There is no special algorithm, the finding is that slowing down the LR or the trunk, while keeping the LR of the experts is enough to eliminate most of the forgetting in the network. You can see in that experiment where chess data was the only thing the model read for 524K characters, yet it kept almost the same performance (i.e. held-out loss) on all other domains. If you keep LR the same across the whole network the loss in other domains degrades dramatically - this is a clear sign of catastrophic forgetting in action. What I can say for sure is that any traditional network that does pose a sign of catastrophic forgetting would not be able to learn any patterns from a single stream of data.
There are no benchmarks published as the model is heavily undertrained, but it is learning. And you can see this clearly in the loss and samples even though they are still barely coherent.
I am not an academic and am not trying to publish a paper about a “major breakthrough” or something like this. I am just a small person who found a cool thing that clearly works and wants to share it with the world. That’s it.
ilaksh 3 hours ago [-]
You can't claim it "works" if it hasn't produced any coherent responses and is still early in your first training attempt.
volotat 3 hours ago [-]
It is a goalpost that is easy to move. By "works" I mean learning from a continuous single (meaning batch-1) stream of data. The fact that it produces full words and full coherent phrases instead of a random stream of characters that would any typical LM produce if trained under the same training regime.
ilaksh 3 hours ago [-]
I would be okay if you shared it as a potential idea and possibly interesting early result, but the language you are actually using to characterize it is misleading or delusional.
Please get a model to the point where it seems like it has some natural language understanding and then share again with reasonable characterization.
volotat 2 hours ago [-]
For sure. As it will pass through the whole corpus I will share the weights, run it through established benchmarks for small models and share all of this as an update. I am also planning on making a Youtube video explaining in detail how it works on a deeper level and the whole reasoning behind why it is built the way it is. But no promises here.
ilaksh 4 hours ago [-]
Actually I'm mad that I wasted my time looking at it based on the claims. He implies it is trained and uses the term "AGI" and "continuous learning". He never finished a single training run or enough that he considers not "undertrained". It's not trained. And actually there is no evidence that it can actually learn anything useful.
synctext 4 hours ago [-]
Indeed this is wasting HN time.
"The model reads 524,000 characters of chess". This is 100KByte of training data in a toy model with rigid parameters and no global learning. Gap with real LLM and trillions of tokens.
This model really addresses the problem of preserving previously learned knowledge, but by restricting the LR of the trunk it stops acquiring new knowledge. Details: "Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective"
xtracto 2 hours ago [-]
This is pretty cool, thanks for sharing. Whe I read it first and saw "continual" I thought for a minute that it was implementing an idea I've been thinking about:
I want to have an agent that thinks continually/non-stop. Imagine a loop of "train of thought" that goes into the LLM and then out. Keep it going so that it "rumiates" thr way we do.
Then, add some sort of "messages" or IRQs when I want to communicate with it. To ask it things and whatnot. I think that sort of cycle in addition to this learning you are doing is what is missing for real AGI.
codethief 25 minutes ago [-]
Some OpenClaw/Hermes-like agents actually work like that. (Can't find the link right now.)
K0balt 4 hours ago [-]
Interesting. I wonder how much could be gained from using tokenization, which makes the model work at a semantic level rather than a syntactic level? I think it’s a force multiplier, but idk if it works here.
bananaflag 6 hours ago [-]
This is the first thing I see in my life that really looks like proto-AGI, it deserves its name.
advael 8 hours ago [-]
Seems interesting, I've been messing with a lot of continuous learning approaches lately and it's cool to see something that's built from the ground up for avoiding catastrophic forgetting. Worth a clone for sure
lostmsu 5 hours ago [-]
It doesn't show any indications of solving catastrophic forgetting.
cpldcpu 8 hours ago [-]
Is this architecture actually able to generalize or is it mostly based on memorization? Have you tried some basic tasks that require generalization? e.g. number addition etc?
volotat 7 hours ago [-]
The model is way too small and undertrained to make any generalization claims. I want to wait until it reads the whole corpus I gave and then test it on some simple established benchmarks to see how it will behave.
jacquesm 7 hours ago [-]
What kind of hardware are you using for training?
nm, I found it:
> RTX 3070 Laptop GPU with 8 GB
Super impressive.
dinfinity 4 hours ago [-]
Seems a bit premature to make an HN post about then, imho.
It's an interesting idea, but it doesn't really do anything interesting yet. I looked at the output in the training run and it is a far, far cry from intelligence. Worse than GPT-2 as it stands.
I do hope it will perform well when scaled and trained, though; best of luck.
Schlagbohrer 2 hours ago [-]
I wish I could understand what a single graph in that nice graphic of graphs meant. No explanation for any vertical or horizontal axis. Looks pretty though.
skeledrew 8 hours ago [-]
Getting conceptually closer to how the human brain works. Looking forward to more of this.
volotat 8 hours ago [-]
I also like how it is very organic. It naturally grows and deletes unused elements, so in addition to traditional backprop there is also a natural selection happening in the background. Each new expert has 16 parents by the way, lol.
hexley19 9 hours ago [-]
Seeing 'Mini-AGI' and '8GB VRAM' in the same sentence is a breath of fresh air. Maybe local AGI isn't so far-fetched.
awfm9 6 hours ago [-]
What's the advantage of doing this, versus becoming good at context management and RAG? I always found trained knowledge unreliable, given that it is lossy by construction.
dinfinity 4 hours ago [-]
Do you find all your own (human) trained knowledge unreliable?
ilusion 7 hours ago [-]
Have you tested what it remembers from early in the stream after a shift in the topics thrown at it?
volotat 7 hours ago [-]
It interleaves random streams of 32K characters long each when reading the whole corpus, but each such stream reads continuously as you would expect. This is a necessary step to prevent just normal, not catastrophic, forgetting. I have not tested it in any other regimes yet with bigger or smaller windows. You can imagine a person that changes the activity from time to time, so I think it is justified. So there is not really "early in the stream".
What I did test though is reading 524K characters of chess data only and see how other domains have degraded. The results are in the readme under "How continual learning works" section. Spoiler: it just barely degraded the performance.
maaaaattttt 5 hours ago [-]
Have you thought about making the whole thing "self-similar"? Every time I hear about MoE I think (and I know it's way easier thought than done) "why stay shallow"?
I mean by that: would it be possible to extend/adapt the architecture so that an expert can be a previously trained Mini-AGI model? And recurse like this? Inuitively I would think some form of generalization could happen, as higher level experts (in the recurrence stack) would become sort of the "intuition" layer.
killerstorm 4 hours ago [-]
Making model to consists of many small modules is inefficient on GPU, especially as routing adds data dependencies, etc, and especially with pytorch (compared to a custom kernel).
The difference might be smaller on a CPU which has limited parallelism.
But it's basically equivalent to a very deep model which might be problematic for training.
comboy 5 hours ago [-]
what character prediction rates are you getting on some unseen datasets?
volotat 5 hours ago [-]
The held-out scores reported in the Readme IS the unseen dataset.
comboy 3 hours ago [-]
These are not successful prediction rate per char though.
volotat 2 hours ago [-]
I got you, will add later to the repository.
comboy 37 minutes ago [-]
Thanks, it seems like a nice way to compare effectiveness of different non-typical methods which are not yet capable of some more ambitious benchmarks.
imtringued 4 hours ago [-]
The expert swapping architecture is very nice. Have you considered doing nested reinforcement learning where you use the nesting as a sort of low pass filter?
The concept is as follows: You train a critic to mimic the datastream and then you train against the critic instead of training against the data. The idea behind this is that the critic will memorize the training data so you do not need to store the full training data anymore. One of the biggest issues with current online stochastic gradient descent is that it is inherently a memory-less technique where the training data acts as the memory.
You can spin this further by going deeper with the nesting and then dropping the supervised critic. I forgot how to put it in words but the goal is that by having a model train against a critic of the critic, you can then drop the top level critic and instead use the mid level critic itself as your meta learning objective to train the actor against an unlabeled data stream.
Top level critic: learns to mimic the labeled training data via online SGD, then you add a simple hand written loss function to compare the predicted output with a given input. Basically you build a model specifically for distillation.
Mid level critic: learns a reward function that mimics the top level critic directly but only gets to see the unlabeled training data and the result of the top level critic.
Actor: The actor is exclusively trained against the mid level critic
Through this concept you end up with the existing training data stored as objective inside the mid level critic so you end up training not only against the latest data but also the already memorized data which should lower catastrophic forgetting. Of course at some point you might need to update the mid level critic again and to avoid that you might get away with just adding a very very wide Linear RNN / State Space Model / Mamba / Gated Delta Net as the middle critic (shower thought: use internal RNN states to represent LoRA vectors).
lostmsu 5 hours ago [-]
This is slop. 8M parameter dense model with context length 64 that you train on enwik9 in 2h will have 1.15 bpb. This model has 1.8 (bits per byte, lower is better).
hanselot 8 hours ago [-]
THANK YOU SO MUCH. This is the missing piece.
loopydosuette 8 hours ago [-]
throwing crumpled paper ball
phatbmt4444 1 hours ago [-]
[flagged]
myshapeprotocol 9 hours ago [-]
[dead]
11 hours ago [-]
bubblegumcrisis 3 hours ago [-]
Very interesting - I have a tangential question.
What motivated you decide to release this. OpenAI or Anthropic will just hoover it up, maybe scale it up and use it if they are interested.
You probably won't know if they do, and the chance they will give you something back is near zero. Why did you release rather than try to scale and build yourself?
(I've been working on some thing, not similar, but not dissimilar in goal - and I just can't get over the fact that tech will steal without giving back)
volotat 2 hours ago [-]
I have no practical means of scaling it up at any compatible scale. I will not make any money on it either way as well. So there is absolutely no reason for hoarding it. And as I said I did use Claude in the process, so Anthropic already has full access to it anyway and could steal it just as easily if they really want to.
I also doubt it is really that valuable on the OpenAI/Anthropic scale, at the same time if people will use it and it will work for them on the personal scale it is already a major win for me. New ideas and optimizations I could never have thought of might bring this up from a toy model to an actually useful model trained locally. Then people could add RL and RLHF and other cool things to it to make it even better.
Rendered at 16:06:09 GMT+0000 (Coordinated Universal Time) with Vercel.
The "trunk learning rate" is set at 0.1x the learning rate for the experts, so learning on different subjects disproportionately happens in the experts, and the trunk portion is comparatively more stable. But the population of experts can grow and shrink:
> The pool grows when it is short of capacity and shrinks when parts of it stop being asked for.
So:
- doesn't the trunk then _eventually_ still undergo catastrophic forgetting, it just may take much longer?
- and before that point, catastrophic forgetting happens in stepwise chunks whenever the expert pool shrinks?
That might explain why there are no benchmarks of any kind.
As a professor who published on continual learning I'm leaning towards agreement[1]. It lacks any substance. No relation to related work, no description of algorithm, no ablation study, just hand-waving that we're feeding some data and "Chess is not forgotten".
This "how-continual-learning-works" markdown text is not an algorithm [2].
[1] https://arxiv.org/abs/2301.12530
[2] https://github.com/volotat/mini-AGI/#how-continual-learning-...
There are no benchmarks published as the model is heavily undertrained, but it is learning. And you can see this clearly in the loss and samples even though they are still barely coherent.
I am not an academic and am not trying to publish a paper about a “major breakthrough” or something like this. I am just a small person who found a cool thing that clearly works and wants to share it with the world. That’s it.
Please get a model to the point where it seems like it has some natural language understanding and then share again with reasonable characterization.
"The model reads 524,000 characters of chess". This is 100KByte of training data in a toy model with rigid parameters and no global learning. Gap with real LLM and trillions of tokens.
This model really addresses the problem of preserving previously learned knowledge, but by restricting the LR of the trunk it stops acquiring new knowledge. Details: "Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective"
I want to have an agent that thinks continually/non-stop. Imagine a loop of "train of thought" that goes into the LLM and then out. Keep it going so that it "rumiates" thr way we do.
Then, add some sort of "messages" or IRQs when I want to communicate with it. To ask it things and whatnot. I think that sort of cycle in addition to this learning you are doing is what is missing for real AGI.
nm, I found it:
> RTX 3070 Laptop GPU with 8 GB
Super impressive.
It's an interesting idea, but it doesn't really do anything interesting yet. I looked at the output in the training run and it is a far, far cry from intelligence. Worse than GPT-2 as it stands.
I do hope it will perform well when scaled and trained, though; best of luck.
What I did test though is reading 524K characters of chess data only and see how other domains have degraded. The results are in the readme under "How continual learning works" section. Spoiler: it just barely degraded the performance.
The difference might be smaller on a CPU which has limited parallelism.
But it's basically equivalent to a very deep model which might be problematic for training.
The concept is as follows: You train a critic to mimic the datastream and then you train against the critic instead of training against the data. The idea behind this is that the critic will memorize the training data so you do not need to store the full training data anymore. One of the biggest issues with current online stochastic gradient descent is that it is inherently a memory-less technique where the training data acts as the memory.
You can spin this further by going deeper with the nesting and then dropping the supervised critic. I forgot how to put it in words but the goal is that by having a model train against a critic of the critic, you can then drop the top level critic and instead use the mid level critic itself as your meta learning objective to train the actor against an unlabeled data stream.
Top level critic: learns to mimic the labeled training data via online SGD, then you add a simple hand written loss function to compare the predicted output with a given input. Basically you build a model specifically for distillation. Mid level critic: learns a reward function that mimics the top level critic directly but only gets to see the unlabeled training data and the result of the top level critic. Actor: The actor is exclusively trained against the mid level critic
Through this concept you end up with the existing training data stored as objective inside the mid level critic so you end up training not only against the latest data but also the already memorized data which should lower catastrophic forgetting. Of course at some point you might need to update the mid level critic again and to avoid that you might get away with just adding a very very wide Linear RNN / State Space Model / Mamba / Gated Delta Net as the middle critic (shower thought: use internal RNN states to represent LoRA vectors).
What motivated you decide to release this. OpenAI or Anthropic will just hoover it up, maybe scale it up and use it if they are interested.
You probably won't know if they do, and the chance they will give you something back is near zero. Why did you release rather than try to scale and build yourself?
(I've been working on some thing, not similar, but not dissimilar in goal - and I just can't get over the fact that tech will steal without giving back)
I also doubt it is really that valuable on the OpenAI/Anthropic scale, at the same time if people will use it and it will work for them on the personal scale it is already a major win for me. New ideas and optimizations I could never have thought of might bring this up from a toy model to an actually useful model trained locally. Then people could add RL and RLHF and other cool things to it to make it even better.