Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best models without these hoops, which makes the case that the average user (=non-criminal) should have access too. As I understood from an ex-Anthropic employee, some orgs got access to Mythos based on their high enough spending level, not on other grounds.
Either we are in command over the software, or the corp is in command over us via the software. I can on a theoretical level understand the concerns, but either we ban all LLMs or we have a level playing field for everybody. Let's not forget: defense and offense are different sides of the same coin in software. I guess this wouldn't apply to bio weapons, but I am not in the know about that.
dantudor 7 minutes ago [-]
There are versions of Qwen3.8-27B that are unrestricted and available from hugging face.
"It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails."
radlad 3 minutes ago [-]
> What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse.
Surely this has unintended side effects on output quality?
binary132 16 minutes ago [-]
Ehh, it’s at least given as the excuse for gain-of-function bioweapon research
VulgarExigency 29 minutes ago [-]
> The first attempt at recovering the key was wrong in a very specific way; it produced a working key and the signature check passed, but a hash the binary computes as an integrity check didn't match. In my experience, most models would have called it done and left it at that, but Qwen 3.8 27B didn't do that. Instead, it highlighted the mismatch, went back to the drawing board, and kept going until the value matched byte for byte.
This seems to be a pattern in the more recently released models that I think accounts for an increase in the quality of their work. They are very persistent in verifying that their work is actually correct, so even if they're not as "smart" as bigger models that get it right the first time, they have the ability to follow through to ensure that the work is actually done.
dataviz1000 4 minutes ago [-]
> They are very persistent in verifying that their work is actually correct
I visualized this with Qwen 3 4B [0] and Sonnet [1] so that people can very easily grok what you mean by "the ability to follow through." What is key about the reasoning tokens is that they will form a pattern of verification tokens in the sequence which, has been shown, will develop in RL training purely without the supervised fine tuning (SFT) which is used to make the output human readable.
I believe this is part of the complaints of new models taking longer/requiring higher spend - they go the extra mile on verification, regardless of whether their change is correct already or not.
So on problems that an earlier model one-shotted an answer to and did some lighter verification, the newer models might take longer to come back to the user due to running all the tests for your software they could find.
catlifeonmars 5 minutes ago [-]
[delayed]
braiamp 22 minutes ago [-]
Well, it seems that Linus doesn't use those:
> And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.
> I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.
> I suspect those things have been trained by people who may not be quite as stubborn as I am.
Why does the screenshot on your pi terminal shows opus-4.6-medium from your claude subscription ? Instead of Qwen ?
kgeist 3 minutes ago [-]
XDA-Developers.com has been posting entirely AI-generated articles lately. Like, a 2026 article about running local models would only talk about Llama 3 and gpt-oss, obviously due to the knowledge cutoff. So I'm not sure if the decompilation in the article actually took place.
MarkWayneNewton 10 minutes ago [-]
The page was created with AI.
saidinesh5 29 minutes ago [-]
Lately I genuinely believe that the future will be large frontier models generating and updating inputs/skills for "good enough" local models to solve our daily problems.
A lot of tasks which need a bit of intelligence don't really need that much compute. Just good enough documentation / skills, tool calling and a good enough local model.
Not sure what exactly this means for all those data centers that are getting built... But exciting times.
catlifeonmars 2 minutes ago [-]
[delayed]
jchw 31 minutes ago [-]
I'd personally like to know more about what tools it used/wanted and the harness setup, because this sounds pretty cool. I have a dual Arc Pro B70 setup and currently get around 22 t/s which isn't great but isn't terrible either (it is at least less quantized.)
I've seen GPT 5.6 Sol happily invoke objdump and even write jobs to run headlessly which Ghidra when trying to disassemble a binary.
trollbridge 22 minutes ago [-]
My M5 Pro gets around 12-15 (6 bit MTP), although I haven’t worked on optimising it at all yet.
A nice thing about running locally is you can run an uncensored model and you don’t have to worry about TOS violations on your OpenAI account when you ask it to “reverse engineer this ancient router firmware and give me a licence key that will work on it”.
RobertasTa 35 minutes ago [-]
Nice writeup — and the 30-minute static-analysis marathon is exactly the kind of task where this model's reasoning earns its keep. Complementary datapoint: I spent this week measuring the thinking levels of the same model locally (27B Q4, Ollama). On a concurrency-bug prompt, thinking off gave a complete answer in ~36s, low/medium thought for a few thousand tokens and answered fine, but high/max spent the entire 16k token budget thinking and never produced an answer at all. So the effort knob is real — deep analysis like this article's job wants it high, but leaving it high for everyday coding just burns context.
Two things surprised me along the way. The defaults differ by stack: llama.cpp's chat template defaults to xhigh while Ollama lands closer to medium, so how much Qwen "overthinks" partly depends on your runtime. And the model card recommends different sampling per mode (temp 1.0 thinking vs 0.7 non-thinking with presence penalty), which almost nobody adjusts when toggling thinking off.
Funny detail: with thinking fully off, my agent harness compensated by just running more tool calls — and still landed the correct fix.
Rendered at 12:37:42 GMT+0000 (Coordinated Universal Time) with Vercel.
Either we are in command over the software, or the corp is in command over us via the software. I can on a theoretical level understand the concerns, but either we ban all LLMs or we have a level playing field for everybody. Let's not forget: defense and offense are different sides of the same coin in software. I guess this wouldn't apply to bio weapons, but I am not in the know about that.
"It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails."
Surely this has unintended side effects on output quality?
This seems to be a pattern in the more recently released models that I think accounts for an increase in the quality of their work. They are very persistent in verifying that their work is actually correct, so even if they're not as "smart" as bigger models that get it right the first time, they have the ability to follow through to ensure that the work is actually done.
I visualized this with Qwen 3 4B [0] and Sonnet [1] so that people can very easily grok what you mean by "the ability to follow through." What is key about the reasoning tokens is that they will form a pattern of verification tokens in the sequence which, has been shown, will develop in RL training purely without the supervised fine tuning (SFT) which is used to make the output human readable.
[0]https://adamsohn.com/reasoning-grid/
[1] https://adamsohn.com/lambda-variance/
> And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.
> I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.
> I suspect those things have been trained by people who may not be quite as stubborn as I am.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
A lot of tasks which need a bit of intelligence don't really need that much compute. Just good enough documentation / skills, tool calling and a good enough local model.
Not sure what exactly this means for all those data centers that are getting built... But exciting times.
I've seen GPT 5.6 Sol happily invoke objdump and even write jobs to run headlessly which Ghidra when trying to disassemble a binary.
A nice thing about running locally is you can run an uncensored model and you don’t have to worry about TOS violations on your OpenAI account when you ask it to “reverse engineer this ancient router firmware and give me a licence key that will work on it”.
Two things surprised me along the way. The defaults differ by stack: llama.cpp's chat template defaults to xhigh while Ollama lands closer to medium, so how much Qwen "overthinks" partly depends on your runtime. And the model card recommends different sampling per mode (temp 1.0 thinking vs 0.7 non-thinking with presence penalty), which almost nobody adjusts when toggling thinking off.
Funny detail: with thinking fully off, my agent harness compensated by just running more tool calls — and still landed the correct fix.