This is a cool idea—I know from snooping on sumbit scripts and node utilization on the HPC that I use at my institution that most submissions leave some compute on the table (and many of them are egregiously bad). I'd probably vote in favor of sending every submitted sbatch script through an LLM (at least for everyone else, I'd would prefer tuning my own usage myself :) ).
Presumably the underlying model here is also an LLM? To what degree is it "fine-tuned", or is it just given a set of tools to build a good picture of cluster usage?
rjpruitt16 1 hours ago [-]
I have been working on open source traffic shaper for agents. I think it may help you better with prediction if requests don’t stampede you
> Datacenters run at roughly 30% to 40% effective utilisation
I wonder what is stopping datacenters from passing this benefit to customers by launching better tuned plans. For example, t series EC2 instances on AWS.
aleksiy123 1 hours ago [-]
Isn’t the fact that you just referenced it indicate that they do?
I feel like it’s probably just complexity.
Different workloads benefit from specific types of optimisations.
keremimo 2 hours ago [-]
Greed
Rendered at 17:04:00 GMT+0000 (Coordinated Universal Time) with Vercel.
Presumably the underlying model here is also an LLM? To what degree is it "fine-tuned", or is it just given a set of tools to build a good picture of cluster usage?
https://www.linkedin.com/posts/rahmi-pruitt-a1bb4a127_agentn...
I wonder what is stopping datacenters from passing this benefit to customers by launching better tuned plans. For example, t series EC2 instances on AWS.
I feel like it’s probably just complexity.
Different workloads benefit from specific types of optimisations.