https://www.reddit.com/r/ClaudeAI/comments/1ukgqwr/looks_lik...

The explanation Anthropic gave for the update doesn't address how the x-axis needed to range up to $50 previously and only $10 now. In any case the pass rates are also lower.

Probably the difference between whatever it is people notice when they say models become "nerfed".

The studies I've seen recently (at least in the software space) put it at something like a 10% increase in coding speed, which for me would probably translate to something like a 3% increase in productivity. I spend a lot more time on things like getting agreement between teams, documenting approaches to things that don't exist on the wiki, etc, that LLMs are significantly less effective at. Or just can't do; no one will be happy if I send an LLM instead of me to meetings.

I suspect a lot of roles are like that. They give a 10-30% boost to the core role function, but that core role is still only 30-50% of what you do.

> that is ~1.5 - 2.5T in value annually

That seems really large, but it's ~2-3x Walmart's yearly revenue, and OpenAI and Anthropic both have estimated valuations that compare to Walmart's market cap. And this is before we consider that they need to do it for cheaper or why would anyone bother. Realistically, potential revenue is probably half that at best.

It's also before cutthroat pricing really kicks in. People are willing to pay for Claude right now; I still suspect that as time goes on people will start looking towards Deepseek/GLM/etc models that provide 95% of the performance at 10% of the price. That'll cut the market even further.

The question is how much demand for knowledge work swells as prices fall, and whether that's a soft landing or a crash.

Minus the cost of inference, that might not be the boon you're making it out to be. I hear what people around here are spending on their api and I'm skeptical that these tools are making me that much more productive.

Personally, for assisted development, I haven't seen much progress in a while.

Pre-bubble pricing: $1400 gets a 128GiB iGPU optimized for inference. Glm and kimi need 800-1000GiB. Call it 1TiB. The $1400 boxes could be ganged into sets of 4-8, with a switch. Call the switch $1000.

Each box has a TDP of 250W. 8 x 250/120V = 16.666A, or one household circuit in the US, so no new power infrastructure is needed.

$1400 x 8+1000=$12,200. Assuming standard five year depreciation, that’s $2440 a year. There are a billion knowledge workers alive today. So that’s $2.4T annual revenue. Average net profit margins on computer hardware are 4.3%. That works out to $105B net income, globally.

So, I guess the question is whether the (currently #2) open weight models provide $1.4-2.4T less value per year than the #1 and #3 models, and, if so, if customers can measure this, or are willing to spend 2x more and deal with censorship, data theft, intentional enshitification, sabotage, ads, product placement, etc, to get the slightly “better” model.

Also, note that my numbers assume moore’s law stopped for all time in 2024, but we’ve seen HW improvements since then.

Pls stop posting you are creating noise.

I think this sort of thinking is a trap, because it presumes that all software has the same constraints.

There's a spectrum of requirements between "chuck this over the wall at Claude, it only has to work once" and "this is a literal rocket ship, formally verify the whole thing".

I've made some things with Claude I don't understand and don't control. It's fine, they're still useful to me. Things for the house that I wasn't going to build manually, some dashboarding stuff and scripts for work, stuff that can crash and burn and I'll be fine.

They won't justify trillions in investment, but they are useful.

Equally, I do agree with you on some things. Sometimes I hand-hold the LLM or forgo it entirely because I want to be 100% sure I know how something works, and can justify a decision if it causes a production outage.

I think the future is probably multiple different tools with different goals. Better IDE integration for some uses, an entirely separate "LLM herd controller" kind of thing for when you're okay with vibe-coding, and the most interesting is something in the middle where you're more in the loop than pure vibe-coding, but don't see the full context like in an IDE. Something where it surfaces changes to key components, but hides things like test changes.

That's a key point. Keeping knowledge and know how inside the company is strategic. For most people GPS did not result in better sense of direction, spellchecking did not help to write without making mistakes, and delegating translation to deepl does help to be better in a foreign languages. I don't see the gain for an individual, a company, a society if a technology reduces the ability to think, do stuff, understand complex problem, working hard at something. Hiring junior also matters, what is boring for a senior dev is useful for a junior, like the "wax on wax off" in Karatekid. Then when the senior dev retired the junior is not junior anymore and the know how is still here. I want to to transfer my knowledge to a junior, not to anthropic or google or openai.

Ideally, working hand in hand with an AI could be like driving a motorcycle vs riding a bicycle. Both are fine, but you go much faster with a motorcycle and you don't lose any ability. But prompting a motorcycle auto-pilot by voice sound a bit stupid and boring. Insane use of energy rarely comes into the equation, which is a bit weird. Personally it is why I am never tempted to use AI. However I see value in AI for finding weakness in a code (inverse of flattery), writing tests with all the edge cases based on specs since tests are often sloppy, asking a fresh view on a very difficult problem. I'd love to hear about the equivalent of move#32 in game 2 (AlphaGo vs Lee Sedol) in a difficult programming task. But I think that massive delegation of code writing is how you lose the knowledge and the know how: what keeps us sharp.

Final word: I asked once a review to claude, the codes involved a db transaction. Nothing complicated, Claude said everything was fine. However the transaction isolation level was not set (I did it on purpose, like if I did not know about isolation levels). He did not ask me if it was my intention to keep the default level. I would have preferred a challenging feedback: why did you chose the default isolation level ? Is it on purpose ? Do you know that the default depend on the db ? Do you know about isolation ? Tell me about the business use case and I'll explain which one would be the best.

Contrary to what some people suggest, I have not hit any maintenance or reliability dead ends. If something breaks, the agent fixes it.

If it cannot, I have the agent instrument the code and work through the logs to check hypotheses, until the source of the issue is found.

If even that would fail, which did not yet happen, I can still do some old fashioned digging and learning, like I always have.

This is for native mobile app development, and the code base is around 100k LOC.

FCFF = EBIT(1-t)-Reinvestment

I dont care about your gross profit - this kind of cash profit determines the value of operating assets.

Chinese models are quickly commodifying frontier inference, the US Gov is preventing domestic SOTA models access to the public and without those models why would consumers still spend $200/month to use the best models?

It’s such a mess and isn’t inspiring confidence as a non-investor.

go ahead m8 we are all waiting... the stage is yours. lets see your model.

<banned>

Not the first one to come up with that likely outcome either. I mean, if you're being restricted from SOTA models now, how long do you expect before the FBI kicks in your door for using an 'illegal' open model?

However the result is exactly the same, concentration of power.

The quite useful tool is to use /opusplan along with /codex:rescue (https://github.com/openai/codex-plugin-cc) means you get quite a strongly reviewed plan using native claude + codex without having to implement the mostly useless trust-me-bro plugins and other bs.

And no (strong) programmer would jump to assuming other people are coding monkeys just because they disagree on what a strong LLM is: that's the kind of thinking reserved for the glorified coding monkeys who wasted their life getting better at writing CRUD apps and are now upset that someone's tooling is dropping the already very low bar there.

(ie. won't feel the need to downvote them just for having yet another crappy AI benchmark)

I only recognize it because I build a product that leaves me looking for information on every major release... and every major release a new crop of folks reply confused about the anomalies on top of anomalies that they're seeing, and they slowly learn this person is just way more unserious than the dogged distribution would imply.

Google Gemini have openly boasted about their animals on vehicles results! https://x.com/JeffDean/status/2024525132266688757

Seniors should be paid to actively introduce juniors to the trade over couple of years. No more bootcamp entry.

And it would be significant $ for senior to agree to expend his time and energy on software engineering apprentices. There would be also very limited number of places with good seniors. Exactly like medicine for a long time now.

In fact it is already happening in some companies I know about - seniors geting their bonuses tied to juniors being under their wings.

If your "force" is above 1 then its ok to have AI power up your force. 2.3 to the power of 3 is 12.169.

But if you're a beginner and your "force" is bellow 1 so power upping this makes it worse. 0.2 to power of 3 is 0.008

If you don’t use a skill, it’s like a gene a species doesn’t need anymore, it will atrophy.

Is that bad and if yes, why? Skill atrophy is not intrinsically bad. I don’t know how to make tinted glas for church windows and I will never learn it because there are machines doing it now.

But I would for example think that critical thinking would be a catastrophic skill atrophy. As far as I know, there is no proven link though (and one would have to define what is “critical thinking” in the first place). Writing assembler without any autocomplete, I’m not so sure it’s such a problematic skill atrophy.

Yes, some skills will atrophy, but the learning curve for LLMs is also steep and you will acquire new skills that will pay off the costs many times over.

We see this in discussions like these where you have people running the gamut from using them as glorified auto-complete or babysitting them (usually a net loss in speed, though it'll feel less draining) via people running multiple agents in several different tabs (a gain) to people prompting for harnesses rather than tasks, and putting the agent in the resulting harness (where the multipliers come in) and even people at the peak of experience with them today are only scratching the surface.

I'm very aware that just as my assembler skills are not what they once were, my skills in the languages I'm now writing less will not be what they once were a few years down this line.

But I produced far more before I started using LLMs through the force multiplier of modern languages and frameworks than I did in assembler in the 80's and 90's, and I produce far more now with LLMs, and I will produce even more in the future by learning how to take advantage of new capabilities.

I have Claude refining a system that wasn't tractable a few years ago in another terminal as I'm writing this. I don't care if it would take me a bit longer to get back up to speed on a C codebase again if I was stripped of all access to LLMs any more than I care if it'd take me a bit longer to get up to speed on programming assembler on a Commodore 64.

I usually start a task with an LLM and then do small refactors using the LLM and then do some manual refactors before I am done. But often for more complex tasks the manual refactors are quite large.

Maybe it is because they can read walls of text so easily, so they output walls of text that are hard to read for humans.

I feel quite sad because a lot of my fellow colleagues are not putting this extra effort in to make things easier to understand by humans. PR review is basically me just doing this extra effort for them and their LLM implementing my comments.

And that is when I can even pinpoint the bad taste in the code structure, sometimes it is not something you can easily describe in a PR comment besides "no human would structure the code like this".

At the end of the day there are goals achieved with coding. Coding is a tool to reach either your business needs or some personal aspiration.

When it comes to businesses I don't think a business cares if you used the best stack possible, or you've written it in assembly, as long as it works. Judging from the biggest coding drivers out there, most of the code produced globally and the biggest apps out there have had skilled engineers writing code but its not always perfect. As long as it works. Lets not forget that the web is build in php and js.

So again my argument is that, are you atrophying a skill that is going to exist in the next 1 to 2 years, or is everything going to shift towards LLM code writting.

Personally I think that LLM code writing is the winner, whether we like it or not, it accelerates business objectives, which at the end of the day its what is the deciding factor.

And yes I do miss the days I was writing code and I was solving complex problems myself.

Could you elaborate on this steep price that you have in mind? What does it consist of?

I have yet to see it, but OK

Either measure it or it sounds like a conspiracy theory

Seniors will be able to stay in the game much longer than before, mark my words.

But they take a lot longer to reach the same goal for complex tasks, so the difference is still very real, and the cost-savings are still very much a question of how well you manage to characterise the tasks they will do quickly and pick and choose what to use when.

I kind of agree that I think the cheap models will eat away at the moat very effectively, but if it doesn't seem more capable to you, you're not giving it complex enough tasks to see what they can do.

(FWIW, I've burned billions of tokens on each of Deepseek, Kimi, GLM5.2, GPT, Sonnet, Opus, Haiku using the same harness, and we've kept stats on cost per task)

Extraordinary thing to say about the fastest growing company in the history of capitalism. They will soon have access to public markets, essentially unlimited capital, and can build insanely large models that they don't have to make public... ever. They can just use those models to run their business, train better models, eat competitors, etc.

But maybe it's Anthropic that isn't thinking ahead enough - you clearly think you can see around corners with your proclamation. So why do you think they have "little to no chance" of surviving long term?

I recently did a fleetwide upgrade to Zig 0.16. Do I remember every single change from 0.15? No. Do I have to? Also no. Both because I can look it up if I need to, but also because the LLM already does.

If I don't look at a codebase that I myself haven't looked at in a year, I will not recognize some things when I return to it. Is this sense of "atrophy" meaningful when this was a problem long before LLMs came on the scene?

Scaling in this sense is not operational (“servers”), but conceptual (“features”).

I don’t want to be a downer but I find many devs are not great at this. Very clever folks, but they tend to not see these issues clearly. They’ll nod and recognize when you talk about separating content from form and the importance of various design principles like high cohesion and loose coupling but completely disregard them once in contact with reality.

Part of the problem, as you nicely showed, is that technology is only a single slice of this problematic pie. Organizations in general are systems as well and they tend to be either badly architected, badly maintained or often both. Some technological issues are downstream from organizational issues and IME those can be become rather dominant variables in the equation and no amount of AI - save full AGI taking control of the company - is going to save us from those factors.

the distinction between personal projects and Enterprise development is a big one. A severe bug in my personal projects, i fix it on the fly. A bug in our products rolled out, nightmare.

Or the largest ad company in the world (Google)?

For now everyone is still sufficiently crap at using AI to need help. We had enough clients trying to build something themselves and then come crying to us.

Open weights models are responsible for enabling reams of research on interpretability methods that do just that. And they have facilitated so much collaboration on architecture, inference optimizations, training and steering methods, and other topics that were completely out of reach with closed models like Anthropic's. It's really staggering to me.

It definitely sounds like the kind of thing that ends the world in B sci-fi thrillers.

The Z.AI is a bit wonky, so now I'm moving to Openrouter for Qwen+Kimi+Deepseek?GLM

My summer project is to figure out a proper agentic system where a "big" model does the planning, but automatically uses a cheaper one for the grunt work. Having Opus to config edits is just stupid :)

What sort of hardware are you using to run local models? And how do you use them?

I might just be having fun with models, but I have actually noticed their capabilities vary somewhat, and so my (perhaps vain) hope is that by using both, one can catch each the other's blindspots. It's still unclear to me if that's consistently happening, but I am making substantial progress in my personal and professional projects, so something seems to be working.

Now that the ban is lifted, max effort Fable 5 is gonna solve this problem quite neatly. Fable to plan and review, Sonnet for the implementation.

Wait, never mind that. Subscribers will only have Fable for a week.

That's not something I understand very well. The less expensive models will quite happily chug away at tasks, if the codebase is well-structured (small files help a lot) and your instructions are clear. In contrast, I've never seen a large model turn bad instructions (instructions that would cause a human to think before starting) into a result I liked. You can run small models almost 10-100x as long for the same price in dollars, which covers a lot of correction and adjustment.

Why does everyone say the trade-offs are rarely worth it?

Alternatively you can draw a horizontal "constant performance" line and see that Opus is cheaper for a given performance level.

The graphs show parts of the cost/performance pareto frontier occupied by Opus 4.8 and others occupied by Sonnet 5.0. If Opus 4.8 was strictly better at cost per task like you say, by definition the entire frontier would be occupied by Opus.

So neither is pareto-dominant over the other. In contrast, Sonnet 5.0 is Pareto-dominent over Sonnet 4.6 on those graphs.

thats true because their point of view makes no sense for us. dario is all in on lesswrong machine god theory and really believes they need to create a super intelligence before anyone else. that means doing as much as possible to slow down others progress and accelerate your own. but the fact that they believe its the only option doesnt make it true for the rest of us.

Are there some Less Wrong posts or similar I should read that probably explain it?

Quite a lot of these models have "safety" (lol) filters in front of them, vs it being heavily encoded into the weights not.

I'm talking back-end, with database models, classes, queries, accompanying front-end layouts, with real dynamic data, running. Stuff that takes days to weeks to spin up, with minimal errors or issues, having cut down on days or weeks of effort, you can focus on testing and making it all into better code.

Sonnet is dumber and more expensive than Opus.

The token efficiency improvements in Opus are missing in Sonnet. Sonnet generates more output tokens and more reasoning tokens.

Any price advantage per token disappears due to volume.

It doesn't make sense to use Sonnet if you have access to Opus.

It feels like your analysis is mostly spot on, it's the confluence of several motivated parties pouring effort into social media.

Many of the posters are pro-foreign models/pro-open source, and most can't distinguish the difference between "open source" and open weight models like Qwen, Minimax, or GLM.

Reminds me of the old "free as in beer" vs "free as in speech" debate. Free beer means you don't pay, but you don't get to see the recipe or change it. Free speech means you get the actual source and the right to study it, modify it, and redistribute it.

Open weight models are basically the beer version. You can download the weights, run them locally, fine-tune them, quantize them, host them on your own boxes — but what you have is a finished product, not the blueprint for how it was built.

Not true

> model that's mostly worse while being more expensive

Not true

> they can ride a wave of misinformation.

Not true