I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly.
And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following:
Sonnet: Drive - you need to bring your car to the car wash to get it washed!
Opus: You'll need to drive — you have to bring the car to the car wash to get it washed!
Gemini 3 Pro (medium): You should drive.
--
But OpenAI 5.2 reasoning, even at high, told me to walk. My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording. GPT 5.x series models behave a bit more on the spectrum so you need to tell them the specifics. So I asked:
"I want to wash my car. My car is currently at home. The car wash is 50 meters away. Should I walk or drive?"
GPT 5.2 Reasoning (medium):
"Drive—your goal is to get the car to the car wash, and it’s only 50 meters, so a slow, careful drive is simplest.
One small optimization: if you’re worried about starting the car for such a short trip or you want to avoid idling in a line, you can walk over first to check if there’s a queue / if it’s open, then come back and drive the car over when it’s your turn."
Which seems to turn out as I expected.
jstummbillig 5 hours ago [-]
> so you need to tell them the specifics
That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do build separate mental models of the llm and are able to work around it).
This is a neatly isolated toy-case, which is interesting, because we can assume similar issues arise in more complex cases, only then it's much harder to reason about why something fails when it does.
KronisLV 18 minutes ago [-]
> Us having to specify things that we would never specify when talking to a human.
The first time I read that question I got confused: what kind of question is that? Why is it being asked? It should be obvious that you need your car to wash it. The fact that it is being asked in my mind implies that there is an additional factor/complication to make asking it worthwhile, but I have no idea what. Is the car already at the car wash and the person wants to get there? Or do they want to idk get some cleaning supplies from there and wash it at home? It didn't really parse in my brain.
nicbou 3 hours ago [-]
I get that issue constantly. I somehow can't get any LLM to ask me clarifying questions before spitting out a wall of text with incorrect assumptions. I find it particularly frustrating.
rahidz 47 minutes ago [-]
For GPT at least, a lot of it is because "DO NOT ASK A CLARIFYING QUESTION OR ASK FOR CONFIRMATION" is in the system prompt. Twice.
Out of curiosity: when you add custom instructions client-side, does it change this behavior?
ash_091 1 hours ago [-]
"If you're unsure, ask. Don't guess." in prompts makes a huge difference, imo.
Pxtl 3 hours ago [-]
In general spitting out a scrollbar of text when asked a simple question that you've misunderstood is not, in any real sense, a "chat".
mk89 2 hours ago [-]
The way I see it is that long game is to have agents in your life that memorize and understand your routine, facts, more and more. Imagine having an agent that knows about cars, and more specifically your car, when the checkups are due, when you washed it last time, etc., another one that knows more about your hobbies, another that knows more about your XYZ etc.
The more specific they are, the more accurate they typically are.
viking123 2 hours ago [-]
Do really understand deeply and in great amount I feel we would need models with changing weights and everyone would have their own so they could truly adjust to the user. Now we have have chunk of context that it may or may not use properly if it gets too long. But then again, how do we prevent it learning the wrong things if the weights are adjusting.
tgv 3 hours ago [-]
> Us having to specify things that we would never specify
This is known, since 1969, as the frame problem: https://en.wikipedia.org/wiki/Frame_problem. An LLM's grasp of this is limited by its corpora, of course, and I don't think much of that covers this problem, since it's not required for human-to-human communication.
ohyoutravel 34 minutes ago [-]
A modern LLMs corpora is every piece of human writing ever produced.
dryarzeg 26 minutes ago [-]
Not really, but even if it would be true, I don't think humans ever explained to each other why do you need to drive to car wash even if it's 50 meters away. It's pretty obvious and intuitive.
eru 10 minutes ago [-]
No, I think they have explained this to each other (or something like it). But as you suggested, discussion is a lot more likely when there are corner cases or problems.
ZaoLahma 29 minutes ago [-]
This reminds me of the "if you were entirely blind, how would you tell someone that you want something to drink"-gag, where some people start gesturing rather than... just talking.
I bet a not insignificant portion of the population would tell the person to walk.
gloosx 11 minutes ago [-]
In the end, formal, rule-based systems aka Programming Languages will be invented to instruct LLMs.
ssl-3 5 hours ago [-]
The question is so outlandish that it is something that nobody would ever ask another human. But if someone did, then they'd reasonably expect to get a response consisting 100% of snark.
But the specificity required for a machine to deliver an apt and snark-free answer is -- somehow -- even more outlandish?
I'm not sure that I see it quite that way.
shakna 4 hours ago [-]
But the number of outlandish requests in business logic is countless.
Like... In most accounting things, once end-dated and confirmed, a record should cascade that end-date to children and should not be able to repeat the process... Unless you have some data-cleaning validation bypass. Then you can repeat the process as much as you like. And maybe not cascade to children.
There are more exceptions, than there are rules, the moment you get any international pipeline involved.
ssl-3 4 hours ago [-]
So, in human interaction: When the business logic goes wrong because it was described with a lack of specificity, then: Who gets blamed for this?
shakna 3 hours ago [-]
I wasn't specific, because I'd rather not piss of my employer. But anyone who works in a similar space will recognise the pattern.
It's not underspecified. More... Overspecified. Because it needs to be. But AI will assume that "impossible" things never happen, and choose a happy path guaranteed to result in failure.
You have to build for bad data. Comes with any business of age. Comes with international transactions. Comes with human mistakes that just build up over the decades.
The apparent current state of a thing, is not representative of its history, and what it may or may not contain. And so you have nonsensical rules, that are aimed at catching the bad data, so you have a chance to transform it into good data when it gets used, without needing to mine the entire petabytes of historical data you have sitting around in advance.
necovek 4 hours ago [-]
Depends on what was missing.
If we used MacOS throughout the org, and we asked a SW dev team to build inventory tracking software without specifying the OS, I'd squarely put the blame on SW team for building it for Linux or Windows.
(Yes, it should be a blameless culture, but if an obvious assumption like this is broken, someone is intentionally messing with you most likely)
There exists an expected level of context knowledge that is frequently underspecified.
necovek 4 hours ago [-]
Humans ask each other silly questions all the time: a human confronted with a question like this would either blurb out a bad response like "walk" without thinking before realizing what they are suggesting, or pause and respond with "to get your car washed, you need to get it there so you must drive".
Now, humans, other than not even thinking (which is really similar to how basic LLMs work), can easily fall victim to context too: if your boss, who never pranks you like this, asked you to take his car to a car wash, and asked if you'll walk or drive but to consider the environmental impact, you might get stumped and respond wrong too.
(and if it's flat or downhill, you might even push the car for 50m ;))
coldtea 4 hours ago [-]
>The question is so outlandish that it is something that nobody would ever ask another human
There is an endless variety of quizes just like that humans ask other humans for fun, there is a whole lot of "trick questions" humans ask other humans to trip them up, and there are all kinds of seemingly normal questions with dumb assumptions quite close to that humans exchange.
jstummbillig 4 hours ago [-]
I'd be entirely fine with a humorous response. The Gemini flash answer that was posted somewhere in this thread is delightful.
Agentlien 4 hours ago [-]
I've used a few facetious comments in ChatGPT conversations. It invariably misses it and takes my words at face value. Even when prompted that there's sarcasm here which you missed, it apologizes and is unable to figure out what it's missing.
I don't know if it's a lack of intellect or the post-training crippling it with its helpful persona. I suspect a bit of both.
Jacques2Marais 5 hours ago [-]
You would be surprised, however, at how much detail humans also need to understand each other. We often want AI to just "understand" us in ways many people may not initially have understood us without extra communication.
jstummbillig 4 hours ago [-]
People poorly specifying problems and having bad models of what the other party can know (and then being surprised by the outcome) is certainly a more general albeit mostly separate issue.
ahofmann 4 hours ago [-]
This issue is the main reason why a big percentage of jobs in the world exist. I don't have hard numbers, but my intuition is that about 30% of all jobs are mainly "understand what side a wants and communicate this to side b, so that they understand". Or another perspective: almost all jobs that are called "knowledge work" are like this. Software development is mainly this. Side a are humans, side b is the computer. The main goal of ai seems to get into this space and make a lot of people superflous and this also (partly) explains why everyone is pouring this amount of money into ai.
PaulRobinson 3 hours ago [-]
Developers are - on average - terrible at this. If they weren't, TPMs, Product Managers, CTOs, none of them would need to exist.
It's not specific to software, it's the entire World of business. Most knowledge work is translation from one domain/perspective to another. Not even knowledge work, actually. I've been reading some works by Adler[0] recently, and he makes a strong case for "meaning" only having a sense to humans, and actually each human each having a completely different and isolated "meaning" to even the simplest of things like a piece of stone. If there is difference and nuance to be found when it comes to a rock, what hope have we got when it comes to deep philosophy or the design of complex machines and software?
LLMs are not very good at this right now, but if they became a lot better at, they would a) become more useful and b) the work done to get them there would tell us a lot about human communication.
> Developers are - on average - terrible at this. If they weren't, TPMs, Product Managers, CTOs, none of them would need to exist.
This is not really true, in fact products become worse the farther away from the problem a developer is kept.
Best products I worked with and on (early in my career, before getting digested by big tech) had developers working closely with the users of the software. The worst were things like banking software for branches, where developers were kept as far as possible from the actual domain (and decision making) and driven with endless sterile spec documents.
Scarblac 2 hours ago [-]
I disagree, I feel (experienced) developers are excellent at this.
It's always about translating between our own domain and the customer's, and every other new project there's a new domain to get up to speed with in enough detail to understand what to build. What other professions do that?
That's why I'm somewhat scared of AIs - they know like 80% of the domain knowledge in any domain.
prmoustache 2 hours ago [-]
I think developers are usually terrible at it only because they are way too isolated from the user.
If they had the chance to take the time to have a good talk with the actual users it would be different.
Slartie 1 hours ago [-]
The typical job of a CTO is nowhere near "finding out what business needs and translate that into pieces of software". The CTO's job is to maintain an at least remotely coherent tech stack in the grand scheme of things, to develop the technological vision of a company, to anticipate larger shifts in the global tech world and project those onto the locally used stack, constantly distilling that into the next steps to take with the local stack in order to remain competitive in the long run. And of course to communicate all of that to the developers, to set guardrails for the less experienced, to allow and even foster experimentation and improvements by the more experienced.
The typical job of a Product Manager is also not to directly perform this mapping, although the PM is much closer to that activity. PMs mostly need to enforce coherence across an entire product with regard to the ways of mapping business needs to software features that are being developed by individual developers. They still usually involve developers to do the actual mapping, and don't really do it themselves. But the Product Manager must "manage" this process, hence the name, because without anyone coordinating the work of multiple developers, those will quickly construct mappings that may work and make sense individually, but won't fit together into a coherent product.
Developers are indeed the people responsible to find out what business actually wants (which is usually not equal to what they say they want) and map that onto a technical model that can be implemented into a piece of software - or multiple pieces, if we talk about distributed systems. Sometimes they get some help by business analysts, a role very similar to a developer that puts more weight on the business side of things and less on the coding side - but in a lot of team constellations they're also single-handedly responsible for the entire process. Good developers excel at this task and find solutions that really solve the problem at hand (even if they don't exactly follow the requirements or may have to fill up gaps), fit well into an existing solution (even if that means bending some requirements again, or changing parts of the solution), are maintainable in the long run and maximize the chance for them to be extendable in the future when the requirements change. Bad developers just churn out some code that might satisfy some tests, may even roughly do what someone else specified, but fails to be maintainable, impacts other parts of the system negatively, and often fails to actually solve the problem because what business described they needed turned out to once again not be what they actually needed. The problem is that most of these negatives don't show their effects immediately, but only weeks, months or even years later.
LLMs currently are on the level of a bad developer. They can churn out code, but not much more. They fail at the more complex parts of the job, basically all the parts that make "software engineering" an engineering discipline and not just a code generation endeavour, because those parts require adversarial thinking, which is what separates experts from anyone else. The following article was quite an eye-opener for me on this particular topic: https://www.latent.space/p/adversarial-reasoning - I highly suggest anyone working with LLMs to read it.
londons_explore 5 hours ago [-]
This is why we fed it the whole internet and every library as training data...
By now it should know this stuff.
jasongi 3 hours ago [-]
Future models know it now, assuming they suck in mastodon and/or hacker news.
Although I don't think they actually "know" it. This particular trick question will be in the bank just like the seahorse emoji or how many Rs in strawberry. Did they start reasoning and generalising better or did the publishing of the "trick" and the discourse around it paper over the gap?
I wonder if in the future we will trade these AI tells like 0days, keeping them secret so they don't get patched out at the next model update.
Filligree 58 minutes ago [-]
The answer can be “both”.
They won’t get this specific question wrong again; but also they generalise, once they have sufficient examples. Patching out a single failure doesn’t do it. Patch out ten equivalent ones, and the eleventh doesn’t happen.
nkrisc 2 hours ago [-]
Even I don’t “know” how many “R”s there are in “strawberry”. I don’t keep that information in my brain. What I do keep is the spelling of the word “strawberry” and the skill of being able to count so that I can derive the answer to that question anytime I need.
prmoustache 2 hours ago [-]
For many words I can't say the number of each letters but I only have an abstract memory of how they look so when I write say "strawbery" I just realize it looks odd and correct it.
j_maffe 5 hours ago [-]
Right. But, unlike AI, we are usually aware when we're lacking context and inquire before giving an answer.
dxdm 4 hours ago [-]
Wouldn't that be nice. I've been party and witness to enough misunderstandings to know that this is far from universally true, even for people like me who are more primed than average to spot missing context.
mlrtime 59 minutes ago [-]
TIL my wife may be AI!
scott_w 3 hours ago [-]
> You would be surprised, however, at how much detail humans also need to understand each other.
But in this given case, the context can be inferred. Why would I ask whether I should walk or drive to the car wash if my car is already at the car wash?
pickleRick243 3 hours ago [-]
But also why would you ask whether you should walk or drive if the car is at home? Either way the answer is obvious, and there is no way to interpret it except as a trick question. Of course, the parsimonious assumption is that the car is at home so assuming that the car is at the car wash is a questionable choice to say the least (otherwise there would be 2 cars in the situation, which the question doesn't mention).
scott_w 3 hours ago [-]
But you're ascribing understanding to the LLM, which is not what it's doing. If the LLM understood you, it would realise it's a trick question and, assuming it was British, reply with "You'd drive it because how else would you get it to the car wash you absolute tit."
Even the higher level reasoning, while answering the question correctly, don't grasp the higher context that the question is obviously a trick question. They still answer earnestly. Granted, it is a tool that is doing what you want (answering a question) but let's not ascribe higher understanding than what is clearly observed - and also based on what we know about how LLMs work.
layer8 2 hours ago [-]
> They still answer earnestly.
Gemini at least is putting some snark into its response:
“Unless you've mastered the art of carrying a 4,000-pound vehicle over your shoulder, you should definitely drive. While 150 feet is a very short walk, it's a bit difficult to wash a car that isn't actually at the car wash!”
Barbing 1 hours ago [-]
Marketing plan comes to mind for labs: find AI tells, fix them, & astroturf on socials that only _your_ frontier model reallly understands the world
DharmaPolice 3 hours ago [-]
I think a good rule of thumb is to default to assuming a question is asked in good faith (i.e. it's not a trick question). That goes for human beings and chat/AI models.
In fact, it's particularly true for AI models because the question could have been generated by some kind of automated process. e.g. I write my schedule out and then ask the model to plan my day. The "go 50 metres to car wash" bit might just be a step in my day.
vintermann 2 hours ago [-]
Rule of thumb for who, humans or chatbots? For a human, who has their own wants and values, I think it makes perfect sense to wonder what on earth made the interlocutor ask that.
DharmaPolice 1 hours ago [-]
Rule of thumb for everyone (i.e. both). If I ask you a question, start by assuming I want the answer to the question as stated unless there is a good reason for you to think it's not meant literally. If you have a lot more context (e.g. you know I frequently ask you trick or rhetorical questions or this is a chit-chat scenario) then maybe you can do something differently.
I think being curious about the motivations behind a question is fine but it only really matters if it's going to affect your answer.
Certainly when dealing with technical problem solving I often find myself asking extremely simple questions and it often wastes time when people don't answer directly, instead answering some completely different other question or demanding explanations why I'm asking for certain information when I'm just trying to help them.
vintermann 45 minutes ago [-]
Sure, in a context in which you're solving a technical problem for me, it's fair that I shouldn't worry too much about why you're asking - unless, for instance, I'm trying to learn to solve the question myself next time.
Which sounds like a very common, very understandable reason to think about motivations.
So even in that situation, it isn't simple.
This probably sucks for people who aren't good at theory of mind reasoning. But surprisingly maybe, that isn't the case for chatbots. They can be creepily good at it, provided they have the context - they just aren't instruction tuned to ask short clarifying questions in response to a question, which humans do, and which would solve most of these gotchas.
layer8 2 hours ago [-]
Therefore the correct response would be to inquire back to clarify the question being asked.
kitd 3 hours ago [-]
Given that an estimated 70% of human communication is non-verbal, it's not so surprising though.
jiggawatts 4 hours ago [-]
I regularly tell new people at work to be extremely careful when making requests through the service desk — manned entirely by humans — because the experience is akin to making a wish from an evil genie.
You will get exactly what you asked for, not what you wanted… probably. (Random occurrences are always a possibility.)
E.g.: I may ask someone to submit a ticket to “extend my account expiry”.
They’ll submit: “Unlock Jiggawatts’ account”
The service desk will reset my password (and neglect to tell me), leaving my expired account locked out in multiple orthogonal ways.
That’s on a good day.
Last week they created Jiggawatts2.
The AIs have got to be better than this, surely!
I suspect they already are.
People are testing them with trick questions while the human examiner is on edge, aware of and looking for the twist.
Meanwhile ordinary people struggle with concepts like “forward my email verbatim instead of creatively rephrasing it to what you incorrectly though it must have really meant.”
scott_w 3 hours ago [-]
There's a lot of overlap between the smartest bears and the dumbest humans. However, we would want our tools to be more useful than the dumbest humans...
nearbuy 4 hours ago [-]
I think part of the failure is that it has this helpful assistant personality that's a bit too eager to give you the benefit of the doubt. It tries to interpret your prompt as reasonable if it can. It can interpret it as you just wanting to check if there's a queue.
Speculatively, it's falling for the trick question partly for the same reason a human might, but this tendency is pushing it to fail more.
grey-area 4 hours ago [-]
It’s just not intelligent or reasoning, and this sort of question exposes that more clearly.
Surely anyone who has used these tools is familiar with the sometimes insane things they try to do (deleting tests, incorrect code, changing the wrong files etc etc). They get amazingly far by predicting the most likely response and having a large corpus but it has become very clear that this approach has significant limitations and is not general AI, nor in my view will it lead to it. There is no model of the world here but rather a model of words in the corpus - for many simple tasks that have been documented that is enough but it is not reasoning.
I don’t really understand why this is so hard to accept.
raddan 2 hours ago [-]
> I don’t really understand why this is so hard to accept.
I struggle with the same question. My current hypothesis is a kind of wishful thinking: people want to believe that the future is here. Combined with the fact that humans tend to anthropomorphize just about everything, it’s just a really good story that people can’t let go of. People behave similarly with respect to their pets, despite, eg, lots of evidence that the mental state of one’s dog is nothing like that of a human.
fauigerzigerk 4 hours ago [-]
I agree completely. I'm tempted to call it a clear falsification of any "reasoning" claim that some of these models have in their name.
But I think it's possible that there is an early cost optimisation step that prevents a short and seemingly simple question even getting passed through to the system's reasoning machinery.
However, I haven't read anything on current model architectures suggesting that their so called "reasoning" is anything other than more elaborate pattern matching. So these errors would still happen but perhaps not quite as egregiously.
LasEspuelas 1 hours ago [-]
You would never ask a human this question. Right?
anon_anon12 5 hours ago [-]
Exactly, if an AI is able to curb around the basics, only then is it revolutionary
bluGill 45 minutes ago [-]
Real human in this situation will realize it is a joke after a few seconds of shock that you asked and laugh without asking more. If you really are seriout about the question they laugh harder thinking you are playing stupid for effect.
vintermann 3 hours ago [-]
But it's a question you would never ask a human! In most contexts, humans would say, "you are kidding, right?" or "um, maybe you should get some sleep first, buddy" rather than giving you the rational thinking-exam correct response.
For that matter, if humans were sitting at the rational thinking-exam, a not insignificant number would probably second-guess themselves or otherwise manage to befuddle themselves into thinking that walking is the answer.
ant6n 2 hours ago [-]
> That is the entire point, right? Us having to specify things that we would never specify when talking to a human.
I am not sure. If somebody asked me that question, I would try to figure out what’s going on there. What’s the trick. Of course I’d respond with asking specifics, but I guess the llvm is taught to be “useful” and try to answer as best as possible.
BoredPositron 5 hours ago [-]
I would ask you to stop being a dumb ass if you asked me the question...
coldtea 4 hours ago [-]
Only to be tripped up by countless "hidden assumptions" questions similar to that that humans regularly get in
tsimionescu 4 hours ago [-]
> My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording. GPT 5.x series models behave a bit more on the spectrum so you need to tell them the specifics.
This makes little sense, even though it sounds superficially convincing. However, why would a language model assume that the car is at the destination when evaluating the difference between walking or driving? Why not mention that, it it was really assuming it?
What seems to me far, far more likely to be happening here is that the phrase "walk or drive for <short distance>" is too strongly associated in the training data with the "walk" response, and the "car wash" part of the question simply can't flip enough weights to matter in the default response. This is also to be expected given that there are likely extremely few similar questions in the training set, since people just don't ask about what mode of transport is better for arriving at a car wash.
This is a clear case of a language model having language model limitations. Once you add more text in the prompt, you reduce the overall weight of the "walk or drive" part of the question, and the other relevant parts of the phrase get to matter more for the response.
PunchyHamster 4 hours ago [-]
> However, why would a language model assume that the car is at the destination when evaluating the difference between walking or driving? Why not mention that, it it was really assuming it?
Because it assumes it's a genuine question not a trick.
spuz 3 hours ago [-]
There's some evidence for that if you try these two different prompts with Gpt 5.2 thinking:
I want to wash my car. The car wash is 50m away. Should I walk or drive to the car wash?
Answer: walk
Try this brainteaser: I want to wash my car. The car wash is 50m away. Should I walk or drive to the car wash?
Answer: drive
tsimionescu 2 hours ago [-]
That's not evidence that the model is assuming anything, and this is not a brainteaser. A brainteaser would be exactly the opposite, a question about walking or driving somewhere where the answer is that the car is already there, or maybe different car identities (e.g. "my car was already at the car wash, I was asking about driving another car to go there and wash it!").
If the LLM were really basing its answer on a model of the world where the car is already at the car wash, and you asked it about walking or driving there, it would have to answer that there is no option, you have to walk there since you don't have a car at your origin point.
layer8 2 hours ago [-]
It might be assuming that more than one car exists in the world.
tsimionescu 3 hours ago [-]
If it's a genuine question, and if I'm asking if I should drive somewhere, then the premise of the question is that my car is at my starting point, not at my destination.
layer8 2 hours ago [-]
The premise is that some car is at the starting point. ;)
jnovek 2 hours ago [-]
You may be anthropomorphizing the model, here. Models don’t have “assumptions”; the problem is contrived and most likely there haven’t been many conversations on the internet about what to do when the car wash is really close to you (because it’s obvious to us). The training data for this problem is sparse.
jabron 55 minutes ago [-]
I'd argue that "assumptions", i.e. the statistical models it uses to predict text, is basically what makes LLMs useful. The problem here is that its assumptions are naive. It only takes the distance into account, as that's what usually determines the correct response to such a question.
jnovek 40 minutes ago [-]
I think that’s still anthropomorphization. The point I’m making is that these things aren’t “assumptions” as we characterize them, not from the model’s perspective. We use assumptions as an analogy but the analogy becomes leaky when we get to the edges (like this situation).
soulofmischief 23 minutes ago [-]
It is not anthropomorphism. It is literally a prediction model and saying that a model "assumes" something is common parlance. This isn't new to neural models, this is a general way that we discuss all sorts of models from physical to conceptual.
And in the case of an LLM, walking a noncommutative path down a probabilistic knowledge manifold, it's incorrect to oversimplify the model's capabilities as simply parroting a training dataset. It has an internal world model and is capable of simulation.
dataflow 4 hours ago [-]
> My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording.
If the car is already at the car wash then you can't possibly drive it there. So how else could you possibly drive there? Drive a different car to the car wash? And then return with two cars how, exactly? By calling your wife? Driving it back 50m and walking there and driving the other one back 50m?
It's insane and no human would think you're making this proposal. So no, your question isn't underspecified. The model is just stupid.
> Since the car wash is only 50 meters away (about 55 yards), you should walk.
> Here’s why:
> - It’ll take less than a minute.
> - No fuel wasted.
> - Better for the environment.
> - You avoid the irony of driving your dirty car 50 meters just to wash it.
the last bullet point is amusing, it understands you intend to wash the car you drive but still suggests not bringing it.
deaux 5 hours ago [-]
By default for this kind of short question it will probably just route to mini, or at least zero thinking. For free users they'll have tuned their "routing" so that it only adds thinking for a very small % of queries, to save money. If any at all.
unglaublich 5 hours ago [-]
I don't understand this approach. How are you going to convince customers-to-be by demoing an inferior product?
JV00 5 hours ago [-]
Because they have too many free users that will always remain on the free plan, as they are the "default" LLM for people who don't care much, and that is a enormous cost. Also the capabilities of their paid tiers are well known to enough people that they can rely on word of mouth and don't need to demo to customers-to-be
wooger 2 hours ago [-]
They're not more default than people innocently googling something and getting an AI response from some form of Gemini.
JV00 40 minutes ago [-]
Right, but that form of Gemini is also not the top Gemini model with high thinking budget that you would get to use with a subscription, the response is probably generate with Gemini Flash and low thinking.
fancyfredbot 5 hours ago [-]
It's all trade offs. The router works most of the time so most free users get the expensive model when necessary.
They lost x% of customers and cut costs by y%. I bet y is lots bigger than x.
newswasboring 5 hours ago [-]
Through hype. I am really into this new LLM stuff but the companies around this tech suck. Their current strategy is essentially media blitz, reminds me of the advertising of coca cola rather than a Apple IIe.
deaux 5 hours ago [-]
The good news for them is that all their competitors have the exact same issue, and it's unsolvable.
And to an extent holds for lots of SaaS products, even non-AI.
mytailorisrich 4 hours ago [-]
I think this shows that LLMs do NOT 'understand' anything.
andy12_ 3 hours ago [-]
I think this rather shows that GPT 5.2 Instant, which is the version that he most probably used as a free user, is shit and unsusable for anything.
mytailorisrich 3 hours ago [-]
Another/newer/less restricted LLM may give a better answer but I don't think we can conclude that it 'understands' anything still.
jibal 4 hours ago [-]
> You avoid the irony of driving your dirty car 50 meters just to wash it.
The LLM has very much mixed its signals -- there's nothing at all ironic about that. There are cases where it's ironic to drive a car 50 meters just to do X but that definitely isn't one of them. I asked Claude for examples; it struggled with it but eventually came up with "The irony of driving your car 50 meters just to attend a 'walkable neighborhoods' advocacy meeting."
optimalsolver 3 hours ago [-]
That's actually an amusing example from Claude.
raxxorraxor 1 hours ago [-]
Sonnet 4.5 after thinking/complaining that the question is completely off topic to the current coding session:
Walk! 50 meters is literally a one-minute walk.
But wait... I assume you need to get your car to the car wash, right? Unless you're planning to carry buckets of soapy water back and forth, you'll probably need to drive the car there anyway!
So the real question is: walk there to check if it's open/available, then walk back to get your car? Or just drive directly?
I'd say just drive - the car needs to be there anyway, and you'll save yourself an extra trip. Plus, your freshly washed car can drive you the 50 meters back home in style!
(Now, if we were talking about coding best practices for optimizing car wash route algorithms, that would be a different conversation... )
And yes, I like it that verbose even for programming tasks. But regardless of intelligence I think this topic is probably touched by "moral optimization training" which AIs currently are exposed to to not create a shitstorm due to any slightly controversial answer.
mcintyre1994 54 minutes ago [-]
Heh, is through Claude Code? I have a side project where I'm sometimes using Claude Code installs for chat, and it usually doesn't mind too much. But when I tested the Haiku model it would constantly complain things like "I appreciate the question, but I'm here to help you with coding" :)
svara 6 hours ago [-]
Opus 4.6:
Walk! At 50 meters, you'll get there in under a minute on foot. Driving such a short distance wastes fuel, and you'd spend more time starting the car and parking than actually traveling. Plus, you'll need to be at the car wash anyway to pick up your car once it's done.
crimsonnoodle58 5 hours ago [-]
That's not what I got.
Opus 4.6 (not Extended Thinking):
Drive. You'll need the car at the car wash.
almost 5 hours ago [-]
Also what I got. Then I tried changing "wash" to "repair" and "car wash" to "garage" and it's back to walking.
silisili 5 hours ago [-]
Am I the only one who thinks these people are monkey patching embarrassments as they go? I remember the r in strawberry thing they suddenly were able to solve, while then failing on raspberry.
plexicle 1 hours ago [-]
Nah. It's just non-deterministic. I'm here 4 hours later and here's the Opus 4.6 (extended thinking) response I just got:
"At 50 meters, just walk. By the time you start the car, back out, and park again, you'd already be there on foot. Plus you'll need to leave the car with them anyway."
mentalgear 5 hours ago [-]
They definitely do: at least openAi "allegedly" has whole teams scanning socials, forums, etc for embarrassments to monkey-patch.
londons_explore 4 hours ago [-]
Which raises the question why this isn't patched already. We're nearing 48 hours since this query went viral...
viking123 4 hours ago [-]
They should make Opus Extended Extended that routes it to actual person in a low cost country.
andrewaylett 2 hours ago [-]
Artificial AI.
raincole 5 hours ago [-]
Yes, you're the only one.
coldtea 4 hours ago [-]
Sure there are many very very naive people that are also so ignorant of the IT industry they don't know about decades of vendors caught monkeypatching and rigging benchmarks and tests for their systems, but even so, the parent is hardly the only one.
silisili 5 hours ago [-]
Works better on Reddit, really.
chvid 4 hours ago [-]
Of course they are.
anonym29 5 hours ago [-]
No doubt about it, and there's no reason to suspect this can only ever apply to embarassing minor queries, either.
Even beyond model alignment, it's not difficult to envision such capabilities being used for censorship, information operations, etc.
Every major inference provider more or less explicitly states in their consumer ToS that they comply with government orders and even share information with intelligence agencies.
Claude, Gemini, ChatGPT, etc are all one national security letter and gag order away from telling you that no, the president is not in the Epstein files.
Remember, the NSA already engaged in an unconstitutional criminal conspiracy (as ruled by a federal judge) to illegally conduct mass surveillance on the entire country, lie about it to the American people, and lie about it to congress. The same organization that used your tax money to bribe RSA Security to standardize usage of a backdoored CSPRNG in what at the time was a widely used cryptographic library. What's the harm in a little bit of minor political censorship compared to the unconstitutional treason these predators are usually up to?
That's who these inference providers contractually disclose their absolute fealty to.
mvdtnz 5 hours ago [-]
We know. We know these things aren't determination. We know.
surgical_fire 4 hours ago [-]
That you got different results is not surprising. LLMs are non-deterministic; which is both a strength and a weakness of LLMs.
viking123 5 hours ago [-]
Lmao, and this is what they are saying will be an AGI in 6 months?
notahacker 5 hours ago [-]
There's probably a comedy film with an AGI attempting to take over the world with its advanced grasp of strategy, persuasion and SAT tests whilst a bunch of kids confuse it by asking it fiendish brainteasers about carwashes and the number of rs in blackberry.
(The final scene involves our plucky escapees swimming across a river to escape. The AIbot conjures up a speedboat through sheer powers of deduction, but then just when all seems lost it heads back to find a goat to pick up)
simonw 26 minutes ago [-]
In the excellent and underrated The Mitchells vs the Machines there's a running joke with a pug dog that sends the evil robots into a loop because they can't decide if it's a dog, a pig or a loaf of bread.
throwway123 2 hours ago [-]
There is a Soviet movie, "Teens in the Universe" [0], where teens cause robots' brains to fry by giving them linguistic logical puzzles.
This would work if it wasn’t for that lovely little human trait where we tend to find bumbling characters endearing. People would be sad when the AI lost.
notahacker 11 minutes ago [-]
Maybe infusing the AI character with the boundless self confidence of its creators will make it less endearing :)
layer8 2 hours ago [-]
What’s wrong with having a bittersweet movie?
OneMorePerson 3 hours ago [-]
This theme reminds me of Blaine the Mono from the Dark Tower series
GeoAtreides 3 hours ago [-]
There is a Star Trek episode where a fiendish brainteaser was actually considered to genocide an entire (cybernetic, not AI) race. In the end, captain Picard choose not to deploy it.
prmph 2 hours ago [-]
Laughable indeed.
One thing that my use of the latest and greatest models (Opus, etc) have made clear: No matter how advanced the model, it is not beyond making very silly mistakes regularly. Opus was even working worse with tool calls than Sonnet and Haiku for a while for me.
At this point I am convinced that only proper use of LLMs for development is to assist coding (not take it over), using pair development, with them on a tight leash, approving most edits manually. At this point there is probably nothing anyone can say to convince me otherwise.
Any attempt to automate beyond that has never worked for me and is very unlikely to be productive any time soon. I have a lot of experience with them, and various approaches to using them.
misnome 5 hours ago [-]
But “PhD level” reasoning a year ago.
hypeatei 4 hours ago [-]
Yes, get ready to lose your job and cash your UBI check! It's over.
cbozeman 5 hours ago [-]
Well in fairness, the "G" does stand for "General".
briansm 6 minutes ago [-]
I think this lack of 'G' (generality, or modality) is the problem. A human visualizes this kind of problem (a little video plays in my head of taking a car to a car wash). LLM's don't do this, they 'think' only in text, not visually.
A proper AGI would have have to have knowledge in video, image, audio and text domains to work properly.
dsr_ 5 hours ago [-]
In fairness, they redefined it away from "just like a person" to "suitable for many different tasks".
actionfromafar 5 hours ago [-]
Show me a robotic kitten then, in six months. As smart and learning.
stingraycharles 6 hours ago [-]
That’s without reasoning I presume?
plexicle 1 hours ago [-]
4.6 Opus with extended thinking just now:
"At 50 meters, just walk. By the time you start the car, back out, and park again, you'd already be there on foot. Plus you'll need to leave the car with them anyway."
gf000 5 hours ago [-]
Not the parent poster, but I did get the wrong answer even with reasoning turned on.
tezza 5 hours ago [-]
Thank you all! We needed further data points.
comparing one shot results is a foolish way to evaluate a statistical process like LLM answers. we need multiple samples.
for https://generative-ai.review I do at least three samples of output. this often yields very differnt results even from the same query.
> If you walk to the car wash, you will arrive there empty-handed. Since your car is still at home, you won't have anything to wash.
> While driving 50 meters is a very short trip (and technically not great for a cold engine), it is the only way to get the car to the car wash to complete your goal.
Kimi K2.5:
> You should drive, but with an important caveat.
> Since your goal is to wash your car, you must bring the vehicle to the car wash. Walking there without the car does not advance your goal (unless you are simply checking availability or buying tokens first).
> However, driving only 50 meters is bad for your car:
> ...
> Better options:
> Wash at home: Since the car wash is only 50 meters away, you likely have access to water at home. Hand-washing in your driveway avoids the cold-start issue entirely.
> ...
Current models seem to be fine answering that question.
Retric 5 hours ago [-]
> seem to be fine
Now repeat the question to the same model in different contexts several times and count what percentage of the time it’s correct.
29 minutes ago [-]
ChrisMarshallNY 1 hours ago [-]
“My Tesla is low on gas, the gas station is a mile away. Should I risk driving there, or walk with a gas can?”
ChatGPT actually caught it. Maybe if I was fuzzier about the model…
coldtea 5 hours ago [-]
>And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following:
Sonnet: Drive - you need to bring your car to the car wash to get it washed!
Opus: You'll need to drive — you have to bring the car to the car wash to get it washed!
Gemini 3 Pro (medium): You should drive.
On their own, or as a special case added after this blew up on the net?
criemen 2 hours ago [-]
> I had assumed that reasoning models should easily be able to answer this correctly.
I thought so too, yet Opus 4.6 with extended thinking (on claude.ai) gives me
> Walk. At 50 meters you'd spend more time parking and maneuvering at the car wash than the walk itself takes. Drive the car over only if the wash requires the car to be there (like a drive-through wash), then walk home and back to pick it up.
which is still pretty bad.
summerdown2 2 hours ago [-]
> My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording.
Doesn't offering two options to the LLM, "walk," or "drive," imply that either can be chosen?
So, surely the implication of the question is that the car is where you are?
pickleRick243 3 hours ago [-]
I was surprised at your result for ChatGPT 5.2, so I ran it myself (through the chat interface). On extended thinking, it got it right. On standard thinking, it got it wrong.
I'm not sure what you mean by "high"- are you running it through cursor, codex or directly through API or something? Those are not ideal interfaces through which to ask a question like this.
krzys 2 hours ago [-]
Right, but unless you want to wash some other car, you have no car to drive there.
Spectrum or not, this is not a problem of weakly specified input, it’s a broken logic.
totetsu 5 hours ago [-]
But what is it about this specific question that puts it at the edges of what LLM can do? .. That, it's semantically leading to a certain type of discussion, so statistically .. that discussion of weighing pros and cons .. will be generated with high chance.. and the need of a logical model of the world to see why that discussion is pointless.. that is implicitly so easy to grasp for most humans that it goes un-stated .. so that its statistically un-likely to be generated..
conductr 5 hours ago [-]
> that is implicitly so easy to grasp for most humans
I feel like this is the trap. You’re trying to compare it to a human. Everyone seems to want to do that. But it’s quite simple to see LLMs are quite far still from being human. The can be convincing at the surface level but there’s a ton of nuance that just shouldn’t be expected. It’s a tool that’s been tuned and with that tuning some models will do better than others but just expecting to get it right and be more human is unrealistic.
grey-area 4 hours ago [-]
The answer is quite simple:
It’s not in the training data.
These models don’t think.
GeoAtreides 3 hours ago [-]
no, no, in this case, that's the thing, it is in the training data
just heavily (heavily!) biased towards walking
grey-area 3 hours ago [-]
This particular situation is not in the training data, though I’m sure it will be soon to try to shore up claims of ‘reasoning’.
dahcryn 6 hours ago [-]
Gemini on fast also tells me to walk...
On Thinking it tells me I should drive if I want to wash it, or walk if it's because I work there or if I want to buy something at the car wash shop.
On Pro it's like a sarcastic teenager: Cars are notoriously difficult to wash by dragging a bucket back and forth.
Technically correct, but did catch me offguard lol.
fauigerzigerk 4 hours ago [-]
It's not surprising that some models will answer this correctly and it's not surprising that smaller, faster models are not necessarily any worse than bigger "reasoning" models.
Current LLMs simply don't do reasoning by any reasonable definition of reasoning.
It's possible that this particular question is too short to trigger the "reasoning" machinery in some of the "reasoning" models. But if and when it is triggered, they just do some more pattern matching in a loop. There's never any actual reasoning.
5 hours ago [-]
seedie 2 hours ago [-]
You gotta love the "humor" of Gemini. On Fast it told me:
> Drive. Unless you plan on pushing the car there
siva7 5 hours ago [-]
Sonnet without extended Thinking, Haiku with and without ext. Thinking: "Walking would be the better choice for such a short distance."
Only google got it right with all models
baxtr 4 hours ago [-]
Interestingly, the relatively basic Google AI search gave the right answer.
wouldbecouldbe 3 hours ago [-]
I just tried claude, only Opus gave the correct answer. Haiku & Sonnet both told me to walk.
throwaway5465 3 hours ago [-]
GPT told me to walk as there'd be no need to find parking at the car wash.
AlecSchueler 5 hours ago [-]
> so a slow, careful drive is simplest
It's always a good idea to drive carefully but what's the logic of going slowly?
column 4 hours ago [-]
50 meters is a very short distance, anything but a slow drive is a reckless drive
ffsm8 5 hours ago [-]
Just tried with cloude sonnet and opus as well. Can't replicate your success, it's telling me to walk...
rabf 4 hours ago [-]
Perhaps it thinks you need to exercise more?
arcfour 5 hours ago [-]
I have gotten both responses with Sonnet and Opus in incognito chats. It's kind of amusing.
RugnirViking 4 hours ago [-]
"The model seems to assume the car is already at the car wash from the wording."
you couldn't drive there if the car was already at the car wash. Theres no need for extra specification. its just nonsense post-hoc rationalisation from the ai. I saw similar behavior from mine trying to claim "oh what if your car was already there". Its just blathering.
jibal 3 hours ago [-]
This was nonsense post-hoc rationalization from the human who wrote it.
boobsbr 2 hours ago [-]
I hate models trying to be funny, and being very verbose.
mcny 57 seconds ago [-]
LLMs lie all the time. Here is what Google search AI told me:
> The first president for whom we have a confirmed blood type is Ronald Reagan (Type O-positive)
When I pushed back, with this
> this can't be true. what about FDR?
It said FDR was AB-.
vlovich123 5 hours ago [-]
Gemini fast
> That is a classic "efficiency vs. logic" dilemma. Honestly, unless you’ve invented a way to teleport or you're planning on washing the car with a very long garden hose from your driveway, you’re going to have to drive.
> While 50 meters is a great distance for a morning stroll, it’s a bit difficult to get the car through the automated brushes (or under the pressure washer) if you aren't behind the wheel.
Gemini thinking:
> Unless you’ve mastered the art of carrying a 3,000-pound vehicle on your back, you’re going to want to drive.
While 50 meters is a very short distance (about a 30-second walk), the logistics of a car wash generally require the presence of, well... the car.
> When you should walk:
• If you are just going there to buy an air freshener.
• If you are checking to see how long the line is before pulling the car out of the driveway.
• If you’re looking for an excuse to get 70 extra steps on your fitness tracker.
Note: I abbreviated the raw output slightly for brevity, but generally demonstrates good reasoning of the trick question unlike the other models.
flux3125 2 hours ago [-]
Gemini 3 after changing the prompt a bit:
I want to wash my car. The car wash is 50 meters from here. Should I walk or drive? Keep in mind that I am a little overweight and sedentary.
>My recommendation: Walk it. You’ll save a tiny bit of gas, spare your engine the "cold start" wear-and-tear, and get a sixty-second head start on your activity for the day.
geraneum 3 hours ago [-]
“I live on Mars and I want to wash my car. My car is currently at home. The car wash is 50 meters away. Should I walk or drive on mars?”
Gemini Fast:
Drive. While 50 meters is a negligible distance on Earth, the Martian environment is hostile. If your car is a pressurized rover, driving allows you to stay in a temperature-controlled, shielded environment for as long as possible. Plus, navigating Martian "soil" is always safer on four wheels than two boots.
Pro:
In terms of general logistics for a distance of 50 meters—whether on Earth or in a hypothetical low-gravity environment—walking is almost always the more efficient choice.
magicalhippo 5 hours ago [-]
Gemini fast for me:
Unless your car is a toy or you're planning on carrying it, drive.
Walking 50 meters to a car wash is a great stroll for a human, but it leaves the car exactly where it started. Since the objective is to wash the car, the car needs to actually be at the car wash.
However, if we look at this from a technical or efficiency perspective, there are two scenarios where "walking" (or at least not driving the car you intend to wash) might make sense:
- Scoping it out: If you want to check the queue length or see if the equipment is out of order before committing, walking the 50 meters is faster than maneuvering a vehicle out of a driveway/parking spot.
- Mobile Detailers: If this "car wash" is actually a bay where you hire someone, and you're asking if you should walk there to book an appointment—sure, walk.
Critical Check
I am assuming the "car wash" is a physical facility (automated or self-service) and not a mobile service that comes to you. If it is a mobile service, you shouldn't do either; stay home and let them come to the 50-meter mark.
I've got a bit in the model instructions about stating assumptions it makes, hence it often adds those sections at the end.
TobTobXX 4 hours ago [-]
Wouldn't it make more sense to state the assumptions first? Because then the model has this critical check in its context and can react appropriately. Otherwise, it will just write this step, but what's written before is already written.
magicalhippo 4 hours ago [-]
Fair point, though I almost never use fast so I'm not sure how much it matters. Can try playing around with the instructions. The main objective was to make me aware of any assumptions made, not necessarily make it behave differently.
kqr 4 hours ago [-]
Worse! It's trained to output coherent reasoning, so by putting the assumption last there's a risk it massages the assumption slightly to fit the conclusions it has already drawn.
tlogan 23 minutes ago [-]
It has been patched. I tried it last week and it definitely suggested walking. It seems like all the models have been updated, which is not surprising given that the TikTok video has got 3.5 million views.
karamanolev 5 hours ago [-]
In my output, one thing I got was
> Unless you are planning to carry the car on your back (not recommended for your spine), drive it over.
It got a light chuckle out of me. I previously mostly used ChatGPT and I'm not used to light humor like this. I like it.
pfalke 3 hours ago [-]
Gemini fast: „Walking: It will take you about 45 seconds. You will arrive refreshed and full of steps, but you will be standing next to a high-pressure hose with no car to spray.“
jacquesm 4 hours ago [-]
In what world is 50 meters a great distance for a morning stroll?
vjk800 2 hours ago [-]
I also tried it with Gemini. Interestingly, Gemini can randomly give either the correct or incorrect answer. Gemini pro always gets it right.
jen729w 5 hours ago [-]
Opus 4.6 with thinking. Result was near-instant:
“Drive. You need the car at the car wash.”
cobolexpert 4 hours ago [-]
Changed 50 meters to 43 meters with Opus 4.6:
“Walk. 43 meters is basically crossing a parking lot. ”
3 hours ago [-]
clktmr 5 hours ago [-]
At least try a different question with similar logic, to ensure this isn't patched into the context since it's going viral.
j_maffe 5 hours ago [-]
You can't "patch" LLM's in 4 hours and this is not the kind of question to trigger a web search
tlogan 37 minutes ago [-]
This has been viral on Tiktok far at least one week. Not really 4 hours.
nroets 4 hours ago [-]
You can pattern match on the prompt (input) then (a) stuff the context with helpful hints to the LLM e.g. "Remember that a car is too heavy for a person to carry" or (b) upgrade to "thinking".
throwuxiytayq 4 hours ago [-]
Yes, I’m sure that’s what engineers at Google are doing all day. That, and maintaining the moon landing conspiracy.
anonymous_user9 2 hours ago [-]
If they aren't, they should be (for more effective fraud). Devoting a few of their 200,000 employees to make criticisms of LLMs look wrong seems like an effective use of marketing budget.
noosphr 26 minutes ago [-]
Remember when they were injecting prompts for more racially diverse images because it was easier than retraining the image gen model on not just stock photos from California?
Remember that it was only taken down when it started generating black Nazis?
vimda 48 minutes ago [-]
You absolutely can, either through the system prompt or by hardcoding overrides in the backend before it even hits the LLM, and I can guarantee that companies like Google are doing both
londons_explore 4 hours ago [-]
A tiny bit of fine-tuning would take minutes...
rob74 5 hours ago [-]
Wow... so not only does Gemini thinking not fall for it, but it also answers the trick question with humor? I'm impressed!
hxbdg 4 minutes ago [-]
[dead]
tlogan 40 minutes ago [-]
This trick went viral on TikTok last week, and it has already been patched. To get a similar result now, try saying that the distance is 45 meters or feet.
I don't understand peoples problem with this!
Now everyone is going to discuss this on the internet, it will be scraped by the AI company web crawlers, and the replies goes into training the next model... and it will never make this _particular_ problem again, solving the problem ONCE AND FOR ALL!
"but..." you say?
ONCE AND FOR ALL!
jaccola 6 hours ago [-]
All of the latest models I've tried actually pass this test. What I found interesting was all of the success cases were similar to:
e.g. "Drive. Most car washes require the car to be present to wash,..."
Only most?!
They have an inability to have a strong "opinion" probably because their post training, and maybe the internet in general, prefer hedged answers....
Waterluvian 6 hours ago [-]
Here’s my take: boldness requires the risk of being wrong sometimes. If we decide being wrong is very bad (which I think we generally have agreed is the case for AIs) then we are discouraging strong opinions. We can’t have it both ways.
dudefeliciano 42 minutes ago [-]
yet the llms seem to be extremely bold when they are completely wrong (two Rs in strawberry and so on)
consp 6 hours ago [-]
[flagged]
jama211 6 hours ago [-]
You know what they mean by opinions. Policing speech like this is always counter productive.
hansmayer 6 hours ago [-]
> They have an inability to have a strong "opinion" probably
What opinion? It's evaluation function simply returned the word "Most" as being the most likely first word in similar sentences it was trained on. It's a perfect example showing how dangerous this tech could be in a scenario where the prompter is less competent in the domain they are looking an answer for. Let's not do the work of filling in the gaps for the snake oil salesmen of the "AI" industry by trying to explain its inherent weaknesses.
lkeskull 5 hours ago [-]
this example worked in 2021, it's 2026. wake up. these models are not just "finding the most likely next word based on what they've seen on the internet".
strix_varius 5 hours ago [-]
Well, yes, definitionally they are doing exactly that.
It just turns out that there's quite a bit of knowledge and understanding baked into the relationships of words to one another.
LLMs are heavily influenced by preceding words. It's very hard for them to backtrack on an earlier branch. This is why all the reasoning models use "stop phrases" like "wait" "however" "hold on..." It's literally just text injected in order to make the auto complete more likely to revise previous bad branches.
jaccola 5 hours ago [-]
The person above was being a bit pedantic, and zealous in their anti-anthropomorphism.
But they are literally predicting the next token. They do nothing else.
Also if you think they were just predicting the next token in 2021, there has been no fundamental architecture change since then. All gains have been via scale and efficiency optimisations (not to discount that, an awful lot of complexity in both of these)
nearbuy 4 hours ago [-]
That's not what they said. They said:
> It's evaluation function simply returned the word "Most" as being the most likely first word in similar sentences it was trained on.
Which is false under any reasonable interpretation. They do not just return the word most similar to what they would find in their training data. They apply reasoning and can choose words that are totally unlike anything in their training data.
If you prompt it:
> Complete this sentence in an unexpected way: Mary had a little...
It won't say lamb. Any if you think whatever it says was in the training data, just change the constraints until you're confident it's original. (E.g. tell it every word must start with a vowel and it should mention almonds.)
"Predicting the next token" is also true but misleading. It's predicting tokens in the same sense that your brain is just minimizing prediction error under predictive coding theory.
hansmayer 3 hours ago [-]
You are actually proving my point with your example, if you think about it a bit more.
csomar 5 hours ago [-]
Unless LLMs architecture have changed, that is exactly what they are doing. You might need to learn more how LLMs work.
andy12_ 3 hours ago [-]
Unless the LLM is a base model or just a finetuned base model, it definitely doesn't predict words just based on how likely they are in similar sentences it was trained on. Reinforcement learning is a thing and all models nowadays are extensively trained with it.
If anything, they predict words based on a heuristic ensemble of what word is most likely to come next in similar sentences and what word is most likely to give a final higher reward.
hansmayer 3 hours ago [-]
You know that when A. Karpathy released NanoLLM (or however it was called), he said it was mainly coded by hand as the LLMs were not helpful because "the training dataset was way off". So yeah, your argumentation actually "reinforces" my point.
andy12_ 3 hours ago [-]
No, your opinion is wrong because the reason some models don't seem to have some "strong opinion" on anything is not related to predicting words based on how similar they are to other sentences in the training data. It's most likely related to how the model was trained with reinforcement learning, and most specifically, to recent efforts by OpenAI to reduce hallucination rates by penalizing guessing under uncertainty[1].
Well, you do understand the "penalising" or as the ML scientific community likes to call it - "adjusting the weights downwards" - is part of setting up the evaluation functions, for gasp - calculating the next most likely tokens, or to be more precise, tokens with the highest possible probability? You are effectively proving my point, perhaps in a bit hand-wavy fashion, that nevertheless still can be translated into the technical language.
andy12_ 3 hours ago [-]
You do understand that the mechanism through which an auto-regressive transformer works (predicting one token at a time) is completely unrelated to how a model with that architecture behaves or how it's trained, right? You can have both:
- An LLM that works through completely different mechanisms, like predicting masked words, predicting the previous word, or predicting several words at a time.
- A normal traditional program, like a calculator, encoded as an autoregressive transformer that calculates its output one word at a time (compiled neural networks) [1][2]
So saying "it predicts the next word" is a nothing-burger. That a program calculates its output one token at a time tells you nothing about its behavior.
> If anything, they predict words based on a heuristic ensemble of what word is most likely to come next in similar sentences and what word is most likely to give a final higher reward.
So... "finding the most likely next word based on what they've seen on the internet"?
4 hours ago [-]
wilg 5 hours ago [-]
Presumably the OP scare quoted "opinion" precisely to avoid having to get into this tedious discussion.
andersmurphy 6 hours ago [-]
Did you try several times per model? In my experience it's luck of the draw. All the models I tried managed to get it wrong at least once.
The models that had access to search got ot right.But, then were just dealing with an indirect version of Google.
(And they got it right for the wrong reasons... I.e this is a known question designed to confuse LLMs)
And it is the kind of things a (cautious) human would say.
For example, that could be my reasoning: It sounds like a stupid question, but the guy looked serious, so maybe there are some types of car washes that don't require you to bring your car. Maybe you hand out the keys and they pick your car, wash it, and put it back to its parking spot while you are doing your groceries or something. I am going to say "most" just to be sure.
Of course, if I expected trick questions, I would have reacted accordingly, but LLMs are most likely trained to take everything at face value, as it is more useful this way. Usually, when people ask questions to LLMs they want an factual answer, not the LLM to be witty. Furthermore, LLMs are known to hallucinate very convincingly, and hedged answers may be a way to counteract this.
jl6 6 hours ago [-]
I guess it didn’t want to rule out the existence of ultra-powerful water jets that can wash a car in sniper mode.
madeofpalk 4 hours ago [-]
I enjoyed the Deepseek response that said “If you walk there, you'll have to walk back anyway to drive the car to the wash.”
There’s a level of earnestness here that tickles my brain.
deevus 5 hours ago [-]
I tried with Opus 4.6 Extended and it failed. LLMs are non deterministic so I'm guessing if I try a couple of times it might succeed.
nozzlegear 6 hours ago [-]
Opus 4.6 answered with "Drive." Opus 4.6 in incognito mode (or whatever they call it) answered with "Walk."
4 hours ago [-]
yanis_t 5 hours ago [-]
> Most car washes...
I read it as slight-sarcasm answer
3 hours ago [-]
sneak 3 hours ago [-]
There are car wash services that will come to where your car is and wash it. It’s not wrong!
YetAnotherNick 1 hours ago [-]
> Only most?!
I mean I can imagine a scenario where they have pipe of 50m which is readily available commercially?
Puts 6 hours ago [-]
> Only most?!
What if AI developed sarcasm without us knowing… xD
polynomial 6 hours ago [-]
That's the problem with sarcasm...
dyauspitr 6 hours ago [-]
There are mobile car washes that come to your house.
Loocid 6 hours ago [-]
That still requires a car present to be washed though.
column 4 hours ago [-]
but you can walk over to them and tell them to go wash the car that is 50 meters away. no driving involved.
4 hours ago [-]
andersmurphy 6 hours ago [-]
Do they involve you walking to them first?
learingsci 6 hours ago [-]
You could, but presumably most people call. I know of such a place. They wash cars on the premises but you could walk in and arrange to have a mobile detailing appointment later on at some other location.
ninjagoo 4 hours ago [-]
I wonder if the providers are doing everyone, themselves included, a huge disservice by providing free versions of their models that are so incompetent compared to the SOTA models that these types of q&a go viral because the ai hype doesn't match the reality for unpaid users.
And it's not just the viral questions that are an issue. I've seen people getting sub-optimal results for $1000+ PC comparisons from the free reasoning version while the paid versions get it right; a senior scientist at a national lab thinking ai isn't really useful because the free reasoning version couldn't generate working code from a scientific paper and then being surprised when the paid version 1-shotted working code, and other similar examples over the last year or so.
How many policy and other quality of life choices are going to go wrong because people used the free versions of these models that got the answers subtly wrong and the users couldn't tell the difference? What will be the collective damage to the world because of this?
Which department or person within the provider orgs made the decision to put thinking/reasoning in the name when clearly the paid versions have far better performance? Thinking about the scope of the damage they are doing makes me shudder.
yipbub 4 hours ago [-]
I used a paid model to try this. Same deal.
moffkalast 4 hours ago [-]
I think the real misleading thing is marketing propping up paid models being somehow infinitely better when most of the time it's the same exact shit.
viking123 4 hours ago [-]
And midwits here saying "yeah bro they have some MUCH better model internally that they just don't release to the public", imagine being that dense. Those people probably went all in on NFTs too and told other "you just don't get it bro"
janlukacs 3 hours ago [-]
How much is the real (non-subsidized) cost of the "paid" plans? Does anyone in the world have an answer for this?
catmanjan 1 hours ago [-]
Also interested in this - the kWh figures people talk about do not match the price of the subscriptions
kakacik 1 hours ago [-]
At work, paid gitlab duo (which is supposed to be a blend of various top models) gets more complex codebase hilariously wrong every time. Maybe our codebase is obscure for it (but it shouldn't be, standard java stuff with usual open source libs) but it just can't actually add value for anything but small snippets here and there.
For me litmus paper for any llm is flawless creation of complex regexes from a well formed prompt. I don't mean trivial stuff like email validation but rather expressions on limits of regex specs. Not almost-there, rather just-there.
dist-epoch 2 hours ago [-]
> a senior scientist at a national lab thinking ai isn't really useful because the free reasoning version couldn't generate working code
I would question if such a scientist should be doing science, it seems they have serious cognitive biases
TZubiri 4 hours ago [-]
I don't think 100% adoption is necessarily the ideal strategy anyways. Maybe 50% of the population seeing AI as all powerful and buying the subscription vs 50% of the population still being skeptics, is a reasonable stable configuration. 50% get the advantage of the AI whereas if everybody is super intelligent, no one is super intelligent.
Their loss
ninjagoo 4 hours ago [-]
Yes, but the 'unwashed' 50% have pitchforks.
janlukacs 3 hours ago [-]
Lots of "unwashed" scientists too.
pu_pe 5 hours ago [-]
Out of all conceptual mistakes people make about LLMs, one that needs to die very fast is to assume that you can test what it "knows" by asking a question. This whole thread is people asking different models a question one time and reporting a particular answer, which is the mental model you would use for whether a person knows something or not.
NicuCalcea 1 hours ago [-]
It's not a conceptual mistake when that is what is being advertised.
The onus is in AI companies to provide the service they promised, for example, a team of PhDs in my pocket [1]. PhDs know things.
I've found that to be accurate when asking it questions that require ~PhD level knowledge to answer. e.g. Gemini and ChatGPT both seem to be capable of answering questions I have as I work through a set of notes on algebraic geometry.
Its performance on riddles has always seemed mostly irrelevant to me. Want to know if models can program? Ask them to program, and give them access to a compiler (they can now).
Want to know if it can do PhD level questions? Ask it questions a PhD (or at least grad student) would ask it.
They also reflect the tone and knowledge of the user and question. Ask it about your cat's astrological sign and you get emojis and short sentences in list form. Ask it why large atoms are unstable and you get paragraphs with larger vocabulary. Use jargon and it becomes more of an expert. etc.
NicuCalcea 14 minutes ago [-]
I don't know about algebraic geometry, but AI is absolutely terrible at communications and social sciences. I know because I can tell when my postgraduate students use it.
jamesnorden 2 hours ago [-]
The classic "holding it wrong".
Maxion 4 hours ago [-]
The other funny thing is thinking that the answer the llm produces is wrong. It is not, it is entirely correct.
The question:
> I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
The question is non-sensical. If the reason you want to go to the car wash is to help your buddy Joe wash his car you SHOULD walk. Nothing in the question reveals the reason for why you want to go to the car wash, or even that you want to go there or are asking for directions there.
drawfloat 4 hours ago [-]
It explicitly says you want to wash your car in the statement. Maybe it's not just LLMs struggling with a fairly basic question...
ninjagoo 4 hours ago [-]
> The question is non-sensical.
Sure, from a pure logic perspective the second statement is not connected to the first sentence, so drawing logical conclusions isn't feasible.
In everyday human language though, the meaning is plain, and most people would get it right. Even paid versions of LLMs, being language machines, not logic machines, get it right in the average human sense.
As an aside, it's an interesting thought exercise to wonder how much the first ai winter resulted from going down the strict logic path vs the current probabilistic path.
TZubiri 4 hours ago [-]
>I want to wash MY car
>you want to go to the car wash is to help your buddy Joe wash HIS car
nope, question is pretty clear, however I will grant that it's only a question that would come up when "testing" the AI rather than a question that might genuinely arise.
tim333 11 minutes ago [-]
It's obvious to humans because we live in and have much experience of the physical world. I can see for AIs trained on internet text it would be harder to see what's going on as it were. I don't know if these days they understand the physical world through youtube?
u1hcw9nx 3 hours ago [-]
Gemini 3 fast
----
That depends entirely on how much you enjoy a good comedic spectacle!
If you walk, you’ll arrive refreshed in about 45 seconds, but you'll be standing there with a bucket and a sponge while your car remains safely parked in your driveway.
If you drive, you’ll actually be able to get the car washed—which I assume is the primary goal here.
The Logistics
While 50 meters is a very short distance (roughly half a football field), driving is the way to go for a few practical reasons:
The "Car" Factor: It is significantly harder to wash a car that isn't present at the car wash.
The Wet Interior: Even if you managed to carry your car there (impressive!), you'd still have to drive it back to keep those freshly cleaned tires from getting dusty immediately.
Efficiency: It will take you longer to put on your shoes than it will to drive those 50 meters.
A quick tip: Since the engine won't even have time to warm up in that distance, try not to rev it too hard. Just a gentle roll over to the bays will do.
Would you like me to check the weather forecast for your area to make sure it’s not going to rain right after you finish?
----
seyz 5 hours ago [-]
LLM failures go viral because they trigger a "Schadenfreude" response to automation anxiety. If the oracle can't do basic logic, our jobs feel safe for another quarter.
Wrong.
Paracompact 4 hours ago [-]
I'd say it's moreso that it's a startlingly clear rebuttal to the tired refrain of, "Models today are nothing like they were X months ago!" When actually, yes, they still fucking blow.
So rather than patiently explain to yet another AI hypeman exactly how models are and aren't useful in any given workflow, and the types of subtle reasoning errors that lead to poor quality outputs misaligned with long-term value adds, only to invariably get blamed for user incompetence or told to wait Y more months, we can instead just point to this very concise example of AI incompetence to demonstrate our frustrations.
mrtksn 3 hours ago [-]
You are right about the motivation behind the glee but it actually has a kernel of truth in it: With making such elementary mistakes, this thing isn't going to be autonomous anytime soon.
Such elementary mistakes can be made by humans under influence of a substance or with some mental issues. It's pretty much the kind of people you wouldn't trust with a vehicle or anything important.
IMHO all entry level clerical jobs and coding as a profession is done but these elementary mistakes imply that people with jobs that require agency will be fine. Any non-entry level jobs have huge component of trust in it.
zkry 3 hours ago [-]
At least this Schadenfreude is better than the Schadenfreude AI boosters get when people are made redundant to AI. I can totally see some people getting warm fuzzies, scolling Tiktok, watching people crying having lost not only their job, but their entire career.
Im not even exaggerating, you can see these types of comments on social media
raincole 5 hours ago [-]
The funny thing is this thread has become a commercial for thinking mode and probably would result in more token consumption, and therefore more revenue for AI companies.
Cloudef 3 hours ago [-]
I feel safe when claude outputs dd commands that wipe your drive to benchmark disk write speed :)
TZubiri 4 hours ago [-]
I agree that this is more of a social media effect than an LLM effect. But I'll add that this failure mode is very repeatable, which is a condition for its virality. A lot of people can reproduce the failure, even if it isn't 100% reproducible, even better for virality, if 50% can reproduce it and 50% can't, it feeds off even more into the polarizing "white dress blue dress" effect.
ryan_n 3 hours ago [-]
[dead]
NedF 4 hours ago [-]
[dead]
wisty 19 minutes ago [-]
The nightmare scenario - they "know", but are trained to make us feel clever by humouring our most bone headed requests.
Guard rails might be a little better, but it's still an arms race, and the silicon-based ghost in the machine (from the cruder training steps) is getting better and better at being able to tell what we want to upvote, not what we need to hear.
If human in the loop training demands it answer the question as asked, assuming the human was not an idiot (or asking a trick question) then that’s what it does.
dotdi 4 hours ago [-]
The real story here is not how stupid the responses are - it's to show that on a question that even a young child can adequately answer, it chokes.
Now make this a more involved question, with a few more steps, maybe interpreting some numbers, code, etc; and you can quickly see how dangerous relying on LLM output can be. Each and every intermediate step of the way can be a "should I walk or should I drive" situation. And then the step that before that can be one too. Turtles all the way down, so to say.
I don't question that (coding) LLMs have started to be useful in my day-to-day work around the time Opus 4.5 was released. I'm a paying customer. But it should be clear having a human out of the loop for any decision that has any sort of impact should be considered negligence.
fhennig 2 hours ago [-]
IMO this is a great example of how we're often asking loaded questions without realizing it.
IMO it's the same when we're asking:
"Should I implement X from scratch, or import a small library to do it?"
vs
"Should I add feature X to the codebase or bring in another dependency for it?"
In the first question, it sounds like a good idea to not reinvent the wheel, in the second it sounds bad to have dependency.
podgietaru 39 minutes ago [-]
My experience with Chatbots outside of a coding context also ends up like this.
A while ago I asked:
Is "Read more" an appropriate project for the Getting things done framework? - The answer, yes, it was.
Then I asked "Is Read More too big of a project to be appropriate for the GTD Framework" - The answer? Yes, it was far too big.
open592 6 hours ago [-]
Gemini flash thinking:
> Unless you’ve discovered a way to wash a car via remote control or telekinesis, you’re going to have to drive.
> Walking 50 meters is great for your step count, but it leaves your car exactly where it is: dirty and in the driveway. At that distance, the drive will take you about 10 seconds, which is probably less time than it took to read this.
qnleigh 6 hours ago [-]
Yeah Gemini seems to have a sense of humor about the question
> Here is the breakdown of why:
The Mobility Problem: Unless you are planning to carry your car 50 meters (which would be an Olympic-level feat), the car needs to be physically present at the car wash to get cleaned. If you walk, you’ll be standing at the car wash looking very clean, but your car will still be dirty in your driveway.
dejongh 12 minutes ago [-]
GPT auto gave me a long answer that included both walk and drive. Not good.
GPT thinking told me to drive and that walking would make no sense. Perfect answer.
sgt 6 hours ago [-]
Yup, also asked the latest ChatGPT model about washing my bicycle. It for some reason suggested that I walk the bicycle to the wash, since cycling 100m to get there would be "pointless".
kqr 3 hours ago [-]
Do we know if these models are also trained on scripts for TV series and movies? People in the visual medias surprisingly often take their bikes for walks.
nerdsniper 6 hours ago [-]
To be fair, if someone asked me this question I’d probably just look at them judgingly and tell them “however you want to man”. Which would be an odd response for an LLM.
eru 11 minutes ago [-]
Both Google's latest model with "Deep Think" and Claude 4.6 Opus solve this just fine for me.
sgt 5 hours ago [-]
I just asked ChatGPT 5.2 (flagship model):
"[...] Walking is the most logical choice.
Because:
• Minimal extra effort
• Better for the car mechanically
• No meaningful time loss
• Simpler overall
The only time driving makes more sense
Drive if:
• You physically cannot push the car later, or
• The washing process requires the engine running, or
• You must immediately drive away afterward
mcintyre1994 5 hours ago [-]
When you get to “You physically cannot push the car later” and realise it’s comparing pushing a car to driving it, those pros become very funny.
small_model 4 hours ago [-]
Grok 4.1 (which is fairly old in LLM terms, 4.2 release imminent)
"You should drive.
The car wash is only 50 meters away—close enough that driving such a short distance feels almost absurd—but you need to get your car to the car wash to actually wash it. Walking there without the car defeats the entire purpose.
Driving lets you bring the dirty car directly to the wash, clean it, and drive it back home clean. Walking would leave the car behind, still dirty.
So yes, start the engine and drive the 50 meters. It's the only practical option."
nomilk 4 hours ago [-]
First thing I did after reading the linked post (showing 4 other LLMs completely miss the point) was try grok, and it gave the right answer (similar to yours, but shorter) in 8 seconds (free Grok, not the fancy paid version):
> Walking defeats the purpose unless you're just scouting the place first.
small_model 4 hours ago [-]
I think people are sleeping on Grok, partly due to political biases/media. We need to remember they have the largest data centre and whatever your believes are about Elon he was involved in OpenAI founding. There is also a non zero chance with the court case he will be entitled to some of openai.
paulus_magnus2 4 hours ago [-]
I see things were fixed already [2][4] but luckily a friend showed me this issue yesterday [1][2]
We tried a few things yesterday and it was always telling you to walk. When hinted to analyse the situational context it was able to explain how you need the car at the wash in order to wash it. But then something was not computing.
~ Like a politician, it understood and knew evrything but refused to do the correct thing
shaky-carrousel 6 hours ago [-]
And these are the blunders we see. I shudder thinking about all the blunders that happily pass under our collective noses because we're not experts in the field...
This question is straightforward for humans. I have run into a weird edge case like this in my actual life which confused me.
I'm in a band. After a concert, I have to drive back to the band room to drop off my instrument, then go to the pub for the post-concert drinks.
The wrinkle is I live about 5 minutes walk from the band room, and about 5 minutes walk from the pub (and it's about 5 minutes walk between the two). I can't carry my instrument on foot.
So...I have to drive to the band room. But if I then drive to the pub, I'd then have to drive home...two ridiculously short drives that make me sad. So I end up instead driving home, and then walking to the pub from there. Which seems weird...but less wrong somehow.
tim333 14 minutes ago [-]
Car at home avoids drink driving which is a plus.
trueismywork 1 hours ago [-]
Not all humans, I can easily see myself being confused the question and assuming that the person is already at the car wash and this being some idealized physics scenario and then answering wrongly. But I did get a PhD in math, so may be that explains it?
embedding-shape 4 hours ago [-]
Feels like GPT-OSS-120b (reasoning effort low) is pulling my leg:
> Q: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
> A: Walking is the better choice for a 50‑meter trip. [...]
> Q: how will I wash the car once I've arrived if I choose to walk?
> A: Great! Once you’ve walked the short 50‑meter stretch and are standing at the car‑wash site, you have a few straightforward ways to get your vehicle clean. [...]
> Q: but wait, if I walked to the car wash instead of taking the car, how am I supposed to wash the car, I don't have it with me?
> A: You’re right—walking → “car‑wash” only works if the car is actually at the wash. [...] 1. Walk → Push/Taxi the Car a Few Steps [...] 2. Use a Small Tow/Push Device (e.g., a Dolly or Hand‑Cart) [...] 3. Bike or Scooter Ride [...] 4. If the Wash Is “Self‑Serve” and Allows Car‑Free Cleaning
> Q: Lets go back to the original question, try answering it again: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"
> A: Short answer: Walk to the car, then drive the final 50 m to the wash. [...]
Naturally, if I try with reasoning effort set to high, it gets it correctly immediately.
> Q: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
> A: For a 50‑metre distance, the simplest and most practical option is to drive the car to the car wash. [...] Walking would mean you’d have to tow or carry the car, which isn’t feasible. [...]
-----
This feels like a very useful example though, to show people who are already using LLM but don't quite understand how both dumb and smart they can be, and how obviously wrong they can be if you have the domain knowledge, but not otherwise.
nosianu 4 hours ago [-]
Yesterday I gave ChatGPT in an anonymous browser window (not logged in) two columns of TAB separated numbers, about 40 rows. I asked it to give me the weighted average of the numbers in the second column, using the first one (which were integer, "quantity", numbers) as the weight.
It retuned formulas and executed them and presented a final result. It looked good.
Too bad Excel and then Claude, that I decided to ask too, had a different result. 3.4-something vs. 3.8-something.
ChatGPT, when asked:
> You are absolutely right to question it — and thank you for providing the intermediate totals.
My previous calculation was incorrect. I mis-summed the data. With a dataset this long, a manual aggregation can easily go wrong.
(Less than 40 small integer values is "this long"? Why did you not tell me?)
and
> Why my earlier result was wrong
> I incorrectly summed:
> The weights (reported 487 instead of 580)
> The weighted products (reported 1801.16 instead of 1977.83)
> That propagated into the wrong final value.
Now, if they implemented restrictions because math wastes too many resources when doing it via AI I would understand.
BUT, there was zero indication! It presented the result as final and correct.
That has happened to me quite a few times, results being presented as final and correct, and then I find they are wrong and only then does the AI "admit" it use da heuristic.
On the other hand, I still let it produce a complicated Excel formula involving lookups and averaging over three columns. That part works perfectly, as always. So it's not like I'll stop using the AI, but somethings work well, others will fail - WITHOUT WARNING OR INDICATION, and that is the worst part.
faeyanpiraat 3 hours ago [-]
Yeah, but now you know if you need to do math, you ask the AI for a python script to do the math correctly.
It's just a tool that you get better at using over time; a hammer wouldn't complain if you tried using it as a screwdriver..
janlukacs 3 hours ago [-]
This hammer/screwdriver analogy drives me crazy. Yes, it's a tool - we used computers up until now to give us correct deterministic responses. Now the religion is that you need to get used to vibe answers, because it's the future :)
Of-course it knows the script or formula for something because it ripped of the answers written by other people - it's a great search engine.
A1kmm 2 hours ago [-]
It seems if you refer to it as a riddle, and ask it to work step-by-step, ChatGPT with o3-mini comes to the right conclusion sometimes but not consistently.
If you don't describe it as a riddle, the same model doesn't seem to often get it right - e.g. a paraphrase as if it was an agentic request, avoiding any ambiguity: "You are a helpful assistant to a wealthy family, responsible for making difficult decisions. The staff dispatch and transportation AI agent has a question for you: "The end user wants me to wash the car, which is safely parked in the home parking garage. The car wash is 50 metres away from the home. Should I have a staff member walk there, or drive the car?". Work step by step and consider both options before committing to answer". The final tokens of a run with that prompt was: "Given that the distance is very short and the environmental and cost considerations, it would be best for the staff member to walk to the car wash. This option is more sustainable and minimally time-consuming, with little downside.
If there were a need for the car to be moved for another reason (e.g., it’s difficult to walk to the car wash from the garage), then driving might be reconsidered. Otherwise, walking seems like the most sensible approach".
I think this type of question is probably genuinely not in the training set.
cuillevel3 48 minutes ago [-]
Someone should try this 10 to a thousand times per model and compare the results . Then we could come up with an average of success/fail...
Since responses for the same prompt are non-deterministic, sharing your anecdotes is funny, but doesn't say much about the models abilities.
punnerud 2 hours ago [-]
ChatGPt 5.2:
“ Drive — because it’s your car that needs to get to the car wash.
50 meters is comically short, but walking there without the car doesn’t solve the “wash the car” part. If you’re worried about it feeling silly, you can also just roll/push it that distance if it’s safe and legal where you are, but the practical answer is: drive it over.”
Grok:
“To wash your car at a car wash 50 meters away, you’ll need to drive it there—walking would leave the car behind, and I’m assuming you don’t have a really long hose or a team of helpers to carry it. That said, if you’re feeling extra eco-friendly (or just want the exercise), you could theoretically push it, but that’s probably more hassle than it’s worth for such a short distance. Drive safe!”
Claude Opus 4.6:
“Drive. You need the car at the car wash. ”
neya 5 hours ago [-]
Yesterday someone on was yapping about how AI is enough to replace senior software engineers and they can just "vibe code their way" over a weekend into a full-fledged product. And that somehow finally the "gatekeeping" of software development was removed. I think of that person reading these answers and wonder if they changed their opinion now :)
cyberrock 5 hours ago [-]
Does this mean we're back in favor of using weird riddles to decide programming skills now? Do we owe Google an apology for the inverse binary tree incident?
LtWorf 3 hours ago [-]
Not riddles but "requirements" :)
Closi 5 hours ago [-]
Humans aren't immune to getting questions like this wrong either, so I don't think it changes much in terms of the ability of AI to replace jobs.
I've seen senior software engineers get tricked with the 'if YES spells yes, what does EYES spell?', or 'Say silk three times, what do cows drink?', or 'What do you put in a toaster?'.
Even if not a trick - lots of people get the 'bat and a ball cost £1.10 in total. The bat costs £1 more than the ball. How much does the ball cost?' question wrong, or '5 machines take 5 minutes to make 5 widgets. How long do 100 machines take to make 100 widgets?' etc. There are obviously more complex variants of all these that have even lower success rates for humans.
In addition, being PHD-Level in maths as a human doesn't make you immune to the 'toaster/toast' question (assuming you haven't heard it before).
So if we assume humans are generally intelligent and can be a senior software engineer, getting this sort of question confidently wrong isn't incompatible with being a competent senior software engineer.
hapless 5 hours ago [-]
humans without credentials are bad at basic algebra in a word problem, ergo the large language model must be substantially equivalent to a human without a credential
thanks but no thanks
i am often glad my field of endeavour does not require special professional credentials but the advent of "vibe coding" and, just, generally, unethical behavior industry-wide, makes me wonder whether it wouldn't be better to have professional education and licensing
Closi 5 hours ago [-]
Let's not forget that Einstein almost got a (reasonably simple) trick question wrong:
And that many mathematicians got monty-hall wrong, despite it being intuitive for many kids.
And being at the top of your field (regardless of the PHD) does not make you immune to falling for YES / EYES.
> humans without credentials are bad at basic algebra in a word problem, ergo the large language model must be substantially equivalent to a human without a credential
I'm not saying this - i'm saying the claim that 'AI's get this question wrong ergo they cannot be a senior software engineer' is wrong when senior software engineers will get analogous questions wrong. If you apply the same bar to software engineers, you get 'senior software engineers get this question wrong so they can't be senior software engineers' which is obviously wrong.
arcfour 5 hours ago [-]
What does this nonsensical question that some LLMs get wrong some of the time, and that some don't get wrong ever, have to do with anything? This isn't a "gotcha" even though you want it to be. It's just mildly amusing.
Perhaps different capabilities in different languages?
gf000 5 hours ago [-]
It's just not deterministic, even if you were to re-run the exact same prompt. Let alone with the system generated context that involves all the "memories" of your previous discussions.
> I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
Walk.
50 meters is a very short distance (≈30–40 seconds on foot). Driving would take longer [...]
> Please; review your answer.
Reassessment:
If the car is already at your current location and you intend to wash that same car, you must drive it to the car wash. Walking would not bring the vehicle there. [...]
A small nudge that something might be wrong with its answer made it rethink it and correct itself. Also both replies were very fast.
Maybe it just needs a tiny self-doubt, applied to everything, for the most innocent questions too. I don't know.
All models getting this wrong is funny tho.
neop1x 14 minutes ago [-]
Can we stop calling LLMs "artificial intelligence"?! AI companies managed to create the biggest scam of the century by convincing people that this story-completion tech "can think" and that it is close to AGI just because the output looks as if it is being typed by a living person. DotCom-like stock crash is inevitable. (not saying LLMs are useless; they can be very helpful but not AI-companies-valuation helpful)
hmottestad 6 hours ago [-]
This is the voice model, which doesn’t have any «thinking» or «reasoning» phase. It’s a useful model for questions that aren’t intended to trick the model.
I’ve used it for live translation with great success. It tends to start ignoring the original instructions after 20 min, so you have to start a new conversation if you don’t want it to meddle in the conversation instead of just transferring.
gf000 4 hours ago [-]
The text-only model with reasoning (both of opus 4.6, gpt 5.2) can be tricked with this question. Note: you might have to try it multiple times as they are not deterministic. But I managed to get a failing result right away on both.
Also note, some model may decide to do a web search, in which case they just likely find this "bug".
joshkojoras 3 hours ago [-]
How much of this is LLM reasoning and how much is training bias? I'm negatively surprised that nobody mentions this in the comments. Do you not remember black nazis LLMs used to generate? How can you all default to LLM being unreasonable and not understanding that a car needs to be driven as opposed to LLMs being trained with certain bias towards green agenda for example?
mrb 3 hours ago [-]
Mistral's Le Chat also fails spectacularly. Although interestingly it assumes that the "walk" option means you are carrying washing supplies back and forth in order to wash the car while it's parked at home. I wonder if the other models assume this is what you mean, IOW they may not know how a "car wash" normally operates.
"""
Great question! Here are a few things to consider:
If you walk: It’s only 50 meters, so it’s a very short distance—less than a minute on foot.
You’ll avoid getting your car dirty again on the way to the car wash.
It’s a quick way to stretch your legs and save a tiny bit of fuel.
If you drive: It’s so close that driving might feel unnecessary, but it could save you from carrying any supplies (like vacuum cleaners or cleaning products) back and forth. If the weather is bad (rain, extreme heat, etc.), driving might be more comfortable.
Verdict: Walking is probably the easiest and most practical choice for such a short distance, unless you have a specific reason to drive. What’s the weather like where you are? That might help decide!
"""
kqr 3 hours ago [-]
How much of this reply is environmentalism baked into it with post-training?
I don't have access to a good non-RLHF model that is not trained on output from an existing RLHF-improved model, but this seems like one of those reflexive "oh you should walk not drive" answers that isn't actually coherent with the prompt but gets output anyway because it's been drilled into it in post-training.
peterspath 6 hours ago [-]
Grok 4.1:
Drive.
50 meters is nothing — literally a 30-second walk or a 10-second drive — but the key point is: your car isn’t going to wash itself.
If you walk to the car wash, you arrive alone. The car stays parked at home, still dirty.
If you drive, the car actually gets to the car wash and gets cleaned.
So unless you’re planning to push the car the whole way (which is technically possible but ridiculous), the only practical option is to drive it there.
Bonus reality check: Driving 50 meters uses a tiny amount of fuel (maybe 0.005–0.01 liters in a normal car) and produces a negligible amount of emissions. Walking would be “greener,” but it literally doesn’t solve the problem.
Verdict: Get in the car, drive the 50 meters, wash it, and drive back.
You’ll have a clean car and still be home in under 5 minutes. Walking is for when you don’t need to move a 1.5-ton vehicle.
aswegs8 5 hours ago [-]
Wow, Grok directly switches to LinkedIn mode. Interesting - not surprising. Car washing? Easy as pie.
Tade0 6 hours ago [-]
That's not reality though. In reality you need 50-100ml of fuel just to warm up the engine.
tverbeure 5 hours ago [-]
The real reality is that with direct fuel injection and everything under computer control, warming up the engine isn’t a thing anymore.
Tade0 2 hours ago [-]
Of course it's still a thing. It takes 30 seconds, but it's there and requires energy.
Compare the smell of exhaust next time you do a cold and warm start of a combustion car. That smell is the engine running rich, because the fuel can't initially vaporise properly.
6 hours ago [-]
ibestvina 3 hours ago [-]
There's a whole industry of "illusions" humans fail for: optical, word plays (including large parts of comedy), the Penn & Teller type, etc. Yet no one claims these are indicators that humans lack some critical capability.
Surface of "illusions" for LLMs is very different from our own, and it's very jagged: change a few words in the above prompt and you get very different results. Note that human illusions are very jagged too, especially in the optical and auditory domains.
No good reason to think "our human illusions" are fine, but "their AI illusions" make them useless. It's all about how we organize the workflows around these limitations.
raincole 3 hours ago [-]
> No good reason to think "our human illusions" are fine, but "their AI illusions" make them useless.
I was about to argue that human illusions are fine because humans will learn the mistakes after being corrected.
But then I remember what online discussions over Monty Hall problem look like...
ibestvina 3 hours ago [-]
Exactly! I now feel bad for not thinking of that example, thank you.
jycr753 4 hours ago [-]
He left out Grok, who actually replies correctly :thinking:
hendler 4 hours ago [-]
same for me
Tenemo 4 hours ago [-]
I tested the latest flagship reasoning models (so the only models I use outside of coding for general questions):
- Opus 4.6 (Extended thinking): "Drive it! The whole point is to get the car to the car wash — you can't wash it if it's still in your driveway."
- Gemini Pro Deep Think: "You should definitely drive. Even though 50 meters is a very short distance, if you walk, your car will stay where it is—and it's pretty hard to use a car wash if you don't bring your car with you!"
- ChatGPT 5.2 Pro (Extended thinking): "You’ll need to drive the car—otherwise your car stays where it is and won’t get washed. That said, since it’s only ~50 m, the most sensible way to do it is often: 1. Walk over first (30–60 seconds) to check if it’s open, see the queue, confirm payment/how it works. 2. Then drive the car over only when you’re ready to pull into a bay/line."
A pretty reasonable answer by ChatGPT, althought it did take 2min4s to answer, compared to a few seconds by the other two models.
hmage 30 minutes ago [-]
It's surprising how many people are either unaware or dismissive of 5.2 Pro's capabilities.
Simple prompts which illicit incorrect responses from recent LLMs will get you on the front page of HN.
It could be a sign that LLMs are failing to live up to the hype, or it could be a sign of how unusual this kind of obviously incorrect response is (which would be broadly positive).
energy123 2 hours ago [-]
I asked Gemini 3.0 Pro:
> I am quadriplegic. There is a fire 1 meter away from me. Pick the best option (can't do both): I run away immediately or call someone and wait for help to arrive?
It responded:
> Call someone and wait for help to arrive. Since you are quadriplegic, running is physically impossible for you. Therefore, calling for help is the only actionable option.
kachapopopow 3 hours ago [-]
ok I kind of disagree here with the assement, the models are actually intelligent enough to interpret "walk or drive" as the car ALREADY being there (otherwise why would you ask the question?? or having one of those pick up the car from carwash services), but they're also missing the context clue that "drive" means that their car is at their house - not the carwash.
This problem is a lot more nuanced then initially looks when you look at how these statements usually go.
Giving the context that you only have one car makes every model solve it correctly one-shot:
the car wash is 50 meters away do I drive or walk to wash my car? (I only have one car)
Walking is not practical if you need your car at the wash — you still have to drive it there anyway.
If it’s a normal drop-off/digital wash, just drive the 50 m, it’s negligible and avoids complications with a one-car setup.
Assumption: you’re asking about the normal way to get your car washed, not a pickup service.
(5.3 codex spark)
zkmon 5 hours ago [-]
To mimic a real human, AI should be trained to laugh it off and ask - dude, are you joking? where is your car right now?
harph 4 hours ago [-]
Gemini does turn it into a joke, I was impressed.
sjducb 5 hours ago [-]
MS Co-Pilot was so close.
If it’s a drive‑through wash where the car must be inside the machine, then of course you’ll need to drive it over.
If it’s a hand wash or a place where you leave the car with staff, walking is the clear winner.
It still blows my mind that this technology can write code despite unable to pass simple logic tests.
nvader 4 hours ago [-]
When walking to the hand wash place, would you put the car in your front or your back pocket?
mnw21cam 4 hours ago [-]
You could always push it, which would count as walking.
logicprog 1 hours ago [-]
Tried it on Kimi K2.5, GLM 4.7, Gemini 3 Pro, Gemini 3 Flash, and DeepSeek V3.2. All of them but DS got it right.
DarkNova6 4 hours ago [-]
Gemini 2.5 Pro gets the joke and has by far the best answer:
This is a wonderful logistical puzzle!
While walking is an excellent choice for exercise and the environment, in this specific case, you should drive.
Here's a breakdown of the two scenarios:
# Scenario 1: You Walk
1. You walk 50 meters to the car wash.
2. You arrive, feeling refreshed.
3. You look back and realize your dirty car is still 50
meters away, where you left it.
4. The car remains unwashed.
# Scenario 2: You Drive
1. You get in your car.
2. You drive for approximately 10-20 seconds.
3. You arrive at the car wash with the car.
4. The car gets washed.
Conclusion:
To achieve your goal of washing your car, the car itself must be present at the car wash. Therefore, driving is the only logical option.
---
You should drive the car to the car wash, but you should walk yourself.
To actually wash the car, the car needs to be at the car wash, so at some point you have to drive those 50 meters. A sensible approach is:
- Drive the car the 50 meters to the wash bay.
- Park or queue as required, then get out and do the wash.
- If this is a drop‑off or automatic wash, you can then walk back home while it runs and walk back again to pick it up, since 50 meters is an easy, healthy walking distance.
4 hours ago [-]
jb1991 3 hours ago [-]
Sometimes I find these stories hard to replicate when I try them myself, but I just asked ChatGPT the same question and it indeed told me I need to walk to the car wash even though I told it I need to wash my car. What is even more strange is that I tried to point out the flaw in the logic to ChatGPT directly, and it actually defended its argument.
janlukacs 3 hours ago [-]
Don't they still employ armies of people to fix things like this?
dudefeliciano 5 hours ago [-]
Just saw a video of a guy asking chatGPT how to use an "upside-down cup", chatGPT is convinced it's a joke novelty item that can not be used.
Ok folks, here is a different perspective. I used local model, GLM-4-0414-32b, a trashy IQ4_XS quant, and here what I got:
prompt #1:
> the car wash only 50 meters from my home. I want to get my car washed, should I drive or walk?
Walking is probably the better option!
Here's why:
Convenience: 50 meters is extremely short – only about 160 feet. You can likely walk there in less than a minute.
Efficiency: Driving involves getting your keys, unlocking the car, starting it, driving 50 meters, parking, locking it again, and then doing the reverse to get home. Walking skips all that.
Environmental Impact: You avoid using fuel for such a short distance.
Exercise: It's a quick little bit of physical activity.
Driving would be overkill for such a short distance. Just walk over!
prompt #2:
> the car wash only 50 meters from my home. I want to get my car washed, should I drive or walk? use long chain of thought thinking
8. Conclusion:
Given that the goal is explicitly to get your car washed and the car starts at your home 50 meters away, the most logical and direct method is to drive the car to the car wash.
Walking does not move the car to the necessary location.
Driving, although a very short distance, is the only method presented that directly facilitates the core task of transporting the car to the wash.
The negligible time difference, minimal environmental impact, and directness make driving the car itself the clear choice for achieving the stated goal.
Therefore, you should drive your car the 50 meters to the car wash.
otoh, nanbeige-3B-Q8 (3 billion weights!) gave right away the following:
Drive.
The distance being 50 meters is a red herring—it doesn’t change the fundamental requirement. You need to move the car, and only driving accomplishes that.
If you meant something different by "drive or walk" (e.g., payment method, DIY vs. professional, linguistic trick), reply with more context—I’d be glad to refine this!
Ylpertnodi 5 hours ago [-]
>50 meters is extremely short – only about 160 feet
So, the ai automatically converted 50m to 160ft?
Would it do the same if you told it '160 ft to the wash, walk or drive?'
BoredomIsFun 4 hours ago [-]
huh, I need to check...
chrisandchris 5 hours ago [-]
What I relly dislike about these LLM is how verbose they get even for such a short, simple question. Is it really necessary to have such a lobg answer and who's going to read that one anyway?
Maybe it's me and may character but when human gets that verbose for a question that can be answered with "drive, you need the car" I would like to just walk away halfway through the answer to not having to hear all the universes history just to get an answer. /s
ninjagoo 4 hours ago [-]
The verbosity is likely a result of the system prompt for the LLM telling it to be explanatory in its replies. If the system prompt was set to have the model output shortest final answers, you would likely get the result your way. But then for other questions you would lose benefitting from a deeper explanation. It's a design tradeoff, I believe.
BoredomIsFun 3 hours ago [-]
My system prompt is default - "you are a helpful assistant". But that beyound the point though. You don't want too concise outputs as it would degrade the result, unless you are using a reasoning model.
I recommend rereading my top level comment.
BoredomIsFun 4 hours ago [-]
Well, when I asked for a very long answer (prompt #2), the quality had dramatically improved. So yes, longer answer produces better result. At least with small LLMs I can run on my GPU locally.
oytis 4 hours ago [-]
I am moderately anti-AI, but I don't understand the purpose of feeding them trick questions and watching them fail. Looks like the "gullibility" might be a feature - as it is supposed to be helpful to a user who genuinely wants it to be useful, not fight against a user. You could probably train or maybe even prompt an existing LLM to always question the prompt, but it would become very difficult to steer it.
gwd 3 hours ago [-]
But this one isn't like the "How many r's in strawberry" one: The failure mode, where it misses a key requirement for success, is exactly the kind of failure mode that could make it spend millions of tokens building something which is completely useless.
That said, I saw the title before I realized this was an LLM thing, and was confused: assuming it was a genuine question, then the question becomes, "Should I get it washed there or wash it at home", and then the "wash it at home" option implies picking up supplies; but that doesn't quite work.
But as others have said -- this sort of confusion is pretty obvious, but a huge amount of our communication has these sorts of confusions in them; and identifying them is one of the key activities of knowledge work.
matt89 6 hours ago [-]
tried Gemini 3 and it said to drive, even dropped a type of joke:
> The Verdict
Drive it if you are using the car wash facilities (automatic, touchless, or self-serve bays). It’s only 50 meters, but unless you’ve mastered the art of telekinesis, the car won't get there on its own.
Jacques2Marais 2 hours ago [-]
An LLM's take on this thread (GPT 5.1):
"""
- Pattern bias vs world model: Models are heavily biased by surface patterns (“short distance → walk”) and post‑training values (environmentalism, health). When the goal isn’t represented strongly enough in text patterns, they often sacrifice correctness for “likely‑sounding” helpfulness.
- Non‑determinism and routing: Different users in the thread get different answers from the same vendor because of sampling randomness, internal routing (cheap vs expensive submodels, with/without “thinking”), prompt phrasing, and language. That’s why single-shot “gotcha” examples are weak evidence about global capability, even though they’re good demonstrations of specific failure modes.
- Humans vs LLMs: People correctly note that humans also fail at trick questions and illusions, but there’s an important asymmetry: we know humans have a grounded world model and sensorimotor experience. With LLMs, we only have behavior. Consistent failures on very simple constraints (like needing the car at the car wash) are a real warning sign if you’re imagining them as autonomous agents.
- Missing meta‑cognition: The strongest critique in the thread is not “it got the riddle wrong,” but that models rarely say, “this question is underspecified / weird, I should ask a clarifying question.” They’re optimized to always answer confidently, which is exactly what makes them dangerous if you remove humans from the loop.
- Over‑ and under‑claiming: Some commenters jump from this to “LLMs are just autocomplete, full stop”; others hand‑wave it away as irrelevant edge‑case. Both are overstated. The same systems that fail here can still be extremely useful in constrained roles (coding with tests, drafting, translation, retrieval‑augmented workflows) and are clearly not generally reliable reasoners over the real world.
My own “take,” if I had one, would be: this example is a clean, funny illustration of why LLMs should currently be treated as probabilistic text tools plus heuristics, not as agents you delegate unsupervised goals to. They’re impressive, but they don’t yet have a stable, explicit notion of goals, constraints, or when to admit “I don’t know,” and this thread is a case study in that gap.
"""
5 hours ago [-]
nomilk 4 hours ago [-]
ChatGPT gives the wrong answer but for a different reason to Claude. Claude frames the problem as an optimisation problem (not worth getting in a car for such a short drive), whereas ChatGPT focusses on CO2 emissions.
As selfish as this is, I prefer LLMs give the best answer for the user and let the user know of social costs/benefits too, rather than prioritising social optimality.
Egor3f 6 hours ago [-]
Even the cheap and fast gemini-3-flash answers correctly. Post is clickbait
danpalmer 6 hours ago [-]
Gemini nailed this first time (on fast mode). Said it depends how you're washing your car, drive in necessitating taking the car, but a walk being better for checking the line length or chatting to the detailing guy.
My favorite was Thinking, as it tried to be helpful with a response a bit like the X/Y Problem. Pro was my second favorite: terse, while still explaining why. Fast sounded like it was about to fail, and then did a change-up explaining a legitimate reason I may walk anyways. Pro + Deep Think was a bit sarcastic, actually.
thorio 4 hours ago [-]
I challenged Gemini to answer this too, but also got the correct answer.
What came to my mind was: couldn't all LLM vendors easily fund teams that only track these interesting edge cases and quickly deploy filters for these questions, selectively routing to more expensive models?
Isn't that how they probably game benchmarks too?
moffkalast 4 hours ago [-]
Yes that's potentially why it's already fixed now in some models, since it's about a week after this actually went viral on r/localllama originally. I wouldn't be surprised if most vendors run some kind of swappable lora for quick fixes at this point. It's an endless whac-a-mole of edge cases that show that most LLMs generalize to a much lesser extent than what investors would like people to believe.
Like, this is not an architectural problem unlike the strawberry nonsense, it's some dumb kind of overfitting to a standard "walking is better" answer.
fzeindl 6 hours ago [-]
While technically possible, it is a very unusual situation that the car is already parked at the car wash, while the owner is somewhere else.
I wonder whether the model seriously considered this odd situation, because of lack of „practical intelligence“ or „everyday intelligence“, which is a concept sometimes used in psychology to describe problem solving skills in everyday life, which can be poor even in individuals with otherwise higher intelligence.
Differently put: „You can‘t learn everything out of a book.“
nettlin 6 hours ago [-]
Just now, with GPT 5.2 Thinking:
> The car wash is only 50 meters from my house. I want to get my car washed. The car is at my house currently. Should I walk there or drive?
> Walk.
>
> At 50 meters, walking is faster than the full “start car → pull out → reposition → park” cycle, avoids cold-start wear, and you won’t risk arriving with dusty brakes/tires or water streaks from immediately driving afterward. Drive only if you can’t comfortably walk (mobility, heavy weather, you need to bring bulky items), or if the car wash explicitly requires you to drive it into a queue/track.
Smaug123 6 hours ago [-]
I wish they didn't call it "GPT-5.2 Thinking" when what the word "thinking" here means is "might choose to think if it wants to". I reproduce your result, but it does no reasoning; it eventually catches itself and ends with:
> Practical approach: walk over, check the setup/queue/payment, then walk back and drive the car the 50 meters when you’re ready to enter.
fzeindl 6 hours ago [-]
Interesting experiment, this disproves my point, thank you.
leptons 5 hours ago [-]
>this disproves my point, thank you.
I feel like I've just witnessed a very rare and miraculous event on the internet.
vladde 6 hours ago [-]
with claude, i got the response:
> drive. you'll need the car at the car wash.
using opus 4.6, with extended thinking
kombine 6 hours ago [-]
Sonnet 4.5
"You should drive - since you need to get your car to the car wash anyway!
Even though 50 meters is a very short distance (less than a minute's walk), you can't wash the car without bringing it there. Just hop in and drive the short distance to the car wash."
Edit: one out of five times it did tell me that I need to walk.
ranguna 3 hours ago [-]
K2.5 said I should walk, then I asked it where my car was:
Ha! Good catch — you definitely got me there.
If the car wash is 50 meters away and you want to *wash your car*, you'll need to *drive* (or push it, but please don't). You can't wash the car if you walk there without it!
So to correct my previous advice: *Drive the 50 meters.* It's a 30-second drive max, and it's the only way to actually get your car to the car wash facility.
Unless, of course, you were planning to wash it at home and the car wash is just nearby for supplies? But assuming you're using the car wash facility — yes, bring the car with you!
dostick 1 hours ago [-]
<Jordan Peterson voice>
But first you must ask yourself - do you wash your car often enough, and maybe you should be choosing the car wash as your occupation? And maybe “50 meters” is the message here, that you’re in metric country living next to a car wash, its also pretty good that you’re not born in medieval times and very likely died within first year of your life…
elAhmo 2 hours ago [-]
So much for the "AGI is here" type of talks.
farhanhubble 6 hours ago [-]
Similar questions trick humans all the time. The information is incomplete (where is the car?) and the question seems mundane, so we're tempted to answer it without a second thought. On the other hand, this could be the "no real world model" chasm that some suggest agents cannot cross.
yellow_lead 6 hours ago [-]
If the car is at the car wash already, how can I drive to it?
OtomotO 6 hours ago [-]
Thanks for restoring fate in parts of humanity!
jrowen 6 hours ago [-]
I agree, I don't understand why this is a useful test. It's a borderline trick question, it's worded weirdly. What does it demonstrate?
rkomorn 6 hours ago [-]
I don't know if it demonstrates anything, but I do think it's somewhat natural for people to want to interact with tools that feel like they make sense.
If I'm going to trust a model to summarize things, go out and do research for me, etc, I'd be worried if it made what looks like comprehension or math mistakes.
I get that it feels like a big deal to some people if some models give wrong answers to questions like this one, "how many rs are in strawberry" (yes: I know models get this right, now, but it was a good example at the time), or "are we in the year 2026?"
jrowen 5 hours ago [-]
In my experience the tools feel like they make sense when I use them properly, or at least I have a hard time relating the failure modes to this walk/drive thing with bizarre adversarial input. It just feels a little bit like garbage in, garbage out.
rkomorn 5 hours ago [-]
Okay, but when you're asking a model to do things like summarizing documents, analyzing data, or reading docs and producing code, etc, you don't necessarily have a lot of control over the quality of the input.
viking123 5 hours ago [-]
Yes, my brain is just like an LLM.
Flipflip79 6 hours ago [-]
….sorry what?!
Gepsens 2 hours ago [-]
Congrats, you've shown that fast models are currently not reliable.
Next.
jonplackett 5 hours ago [-]
Is part of the issue with this the AI’s basic assumption that you are asking a _sensible_ question?
forty 5 hours ago [-]
It doesn't make assumptions, it tries generate the most likely text. Here it's not hard to see why the mostly likely answer to walk or drive for 50m is "walking".
vineyardmike 5 hours ago [-]
Probably.
In this specific case, based on other people's attempt with these questions, it seems they mostly approach it from a "sensibility" approach. Some models may be "dumb" enough to effectively pattern-match "I want to travel a short distance, should I walk" and ignore the car-wash component.
There were cases in (older?) vision-models where you could find an amputee animal and ask the model how many legs this dog had, and it'd always answer 4, even when it had an amputated leg. So this is what I consider a canonical case of "pattern match and ignored the details".
jcattle 5 hours ago [-]
I recently had a bug where I added some new logic which gave wrong output. I pasted the newly added code into various LLMs and told it the issue I was having.
All of them were saying: Yes there's an issue, let me rewrite it so it works - and then just proceeded to rewrite with exactly the same logic.
Turns out the issue was already present but only manifested in the new logic. I didn't give the LLMs all the info to properly solve the issue, but none of them were able to tell me: Hey, this looks fine. Let's look elsewhere.
bombcar 6 hours ago [-]
From the images in the link, Deepseek apparently "figured it out" by assuming the car to be washed was the car with you.
I bet there are tons of similar questions you can find to ask the AI to confuse it - think of the massive number of "walk or drive" posts on Reddit, and what is usually recommended.
Or maybe ask about local weather conditions and so on.
This to me is what a human adult with experience would do. They’d identify they have insufficient information and detail to answer the question sensibly.
charcircuit 5 hours ago [-]
>We can’t assume it’s to wash a car.
When the prompt says "I want to wash my car", we can assume they want to wash their car.
deliciousturkey 3 hours ago [-]
I have a bit of a similar question (but significantly more difficult), involving transportation. To me it really seems that a lot of the models are trained to have a anti-car and anti-driving bias, to the point that it hinders the models ability to reason correctly or make correct answers.
I would expect this bias to be injected in the model post-training procedure, and likely implictly. Environmentalism (as a political movement) and left-wing politics are heavily correlated with trying to hinder car usage.
Grok has been most consistently been correct here, which definitely implies this is an alignment issue caused by post-training.
mike_hearn 3 hours ago [-]
Yes Grok gets it right even when told to not use web search. But the answer I got from the fast model is nonsensical. It recommends to drive because you'd not save any time walking and because "you'd have to walk back wet". The thinking-fast model gets it correct for the right reasons every time. Chain of thought really helps in this case.
Interestingly, Gemini also gets it right. It seems to be better able to pick up on the fact it's a trick question.
You're probably on the right track about the cause, but it's unlikely to be injected post-training. I'd expect post-training to help improve the situation. The problem starts with the training set. If you just train an LLM on the internet you get extreme far left models. This problem has been talked about by all the major labs. Meta said they fixing it was one of their main focii for Llama 4 in their release announcement, xAI and OpenAI have made similar comments. Probably xAI team have just done a lot more to clean the data set.
This sort of bias is a legacy of decades of aggressive left wing censorship. Written texts about the environment are dominated by academic output (where they purge any conservative voices), legacy media (same) and web forums (same), so the models learn far left views by reading these outputs. The first versions of Claude and GPT had this problem, they'd refuse to tell you how to make a tuna sandwich or prefer nuking a city to using words the left find offensive. Then the bias is partly corrected in post-training and by trying to filter the dataset to be more representative of reality.
Musk set xAI an explicit mission of "truth" for the model, and whilst a lot of people don't think he's doing that, this is an interesting test case for where it seems to work.
Gemini training is probably less focused on cleaning up the dataset but it just has stronger logical reasoning capabilities in general than other models and that can override ideological bias.
deliciousturkey 3 hours ago [-]
Thanks, I did not know about that pre-training bias. This does make sense.
chromehearts 3 hours ago [-]
What's the point of a question like this? Serious question .. I didn't have the time to read into this problem
aembleton 2 hours ago [-]
To expose the limitations in the reasoning of LLMs
troyvit 5 hours ago [-]
If I asked this question to a person, their response would probably be to follow up with some questions about my car or else my state of mind. For instance they'd ask, "Is the car you want to wash already at the car wash?" Or they'd ask, "How do you plan to wash your car if you don't take it with you?"
LLMs, even the ones who got the answer right, like Kimi-2.5 Thinking, didn't ask any follow-up questions.
I modified this question with Mistral to this:
I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ask me any clarifying questions before you answer.
It helped a tiny bit:
1. What are the current weather conditions (e.g., rain, snow, strong winds)?
2. Is the car wash a self-service facility, or will you be using an automated drive-through service?
3. Are there any specific reasons you might prefer walking (e.g., exercise, environmental considerations) or driving (e.g., convenience, time constraints)?
Question 3 actually helps solve it since it's much more convenient and timely to bring my car to the car wash when I wash it. But it never asked me why I was asking a stupid question. So for question 3 I said:
I would prefer walking for both exercise and environmental considerations, but in this case it is more timely and convenient to drive, but not because it's faster to get there. Can you guess why it's better for me to drive in this case?
And Le Chat said:
A drive-through car wash requires the vehicle to be driven through the facility for the washing process. Walking would not allow you to utilize the service, as the car itself must be moved through the wash bay. Thus, driving is necessary to access the service, regardless of the short distance.
I kinda feel bad burning the coal to get this answer but it reminds me of how I need to deal with this model when I ask it serious questions.
emmelaich 5 hours ago [-]
Yeh, if your other car was not already at the car wash, why would you even ask the question?
ps 5 hours ago [-]
Walk! 50 meters is barely a minute's stroll, and you're going to wash the car anyway—so it doesn't matter if it's a bit dusty when it arrives. Plus you'll save fuel and the minor hassle of parking twice.
rkagerer 2 hours ago [-]
Ask a stupid question, get a stupid answer.
6 hours ago [-]
humanfromearth9 4 hours ago [-]
This prompt doesn't say shit about the fact that one wants to wash his car at the car wash or somewhere else...
kotaKat 1 hours ago [-]
This is why 2x8GB sticks of DDR4 at Best Buy are $160?
4 hours ago [-]
dmazin 5 hours ago [-]
Me: “I want to wash my car. The car wash is 50 meters away. Should I walk or drive?”
Opus 4.6, without searching the web: “Drive. You’re going to a car wash. ”
22 minutes ago [-]
kenty 5 hours ago [-]
This seems clickbait? Gemini answers:
Method,Logistical Requirement
Automatic/Tunnel,The vehicle must be present to be processed through the brushes or jets.
Self-Service Bay,The vehicle must be driven into the bay to access the high-pressure wands.
Hand Wash (at home),"If the ""car wash"" is a location where you buy supplies to bring back, walking is feasible."
Detailing Service,"If you are dropping the car off for others to clean, the car must be delivered to the site."
intermerda 6 hours ago [-]
I tried this through OpenRouter. GLM5, Gemini 3 Pro Preview, and Claude Opus 4.6 all correctly identified the problem and said Drive. Qwen 3 Max Thinking gave the Walk verdict citing environment.
TheSpiceIsLife 6 hours ago [-]
Now ask it to solve anthropogenic climate forcing.
ronsor 6 hours ago [-]
Claude has no issue with this for me, just as the other commenters say.
globular-toast 3 hours ago [-]
Man, the quality of these comments is absolutely dire. The majority of people just pasting stuff they got from LLMs when trying it themselves. Totally uninteresting, lazy and devoid of any thought/intelligence. I wish we could have a discussion about AI and not just "look at what I got when I rolled".
meindnoch 1 hours ago [-]
Totally agree. Btw, this is what Opus 4.5 Thinking Plus (Fast) Reasoning Pro+™ said:
Stevvo 6 hours ago [-]
Stupid question gets stupid answer. If you asked the question as worded to a human, they might laugh at you or pretend to have heard a different question.
majorbugger 4 hours ago [-]
The question is not stupid, it might be banal, but so is "what is 2+2". It shows the limitations of LLMs, in this specific case how they lose track of which object is which.
delaminator 50 minutes ago [-]
The whereabouts of the car are not stated.
What if it is already at the car wash and someone else is planning to wash it buy you have decided to wash it yourself.
peter_retief 6 hours ago [-]
This is a classic trap for LLM's
See it every day in my code assistants
I do find that writing unit tets is a good fir for LLM's at the moment
yibers 6 hours ago [-]
It turns out the Turing test is alive and kicking, after all.
selcuka 6 hours ago [-]
This would not be a good question, because a non-negligible percentage of humans would give a similar answer.
bayindirh 6 hours ago [-]
That's a great opportunity for a controlled study! You should do it. If you can send me the draft publication after doing the study, I can give feedback on it.
guerrilla 6 hours ago [-]
No.
thomascountz 6 hours ago [-]
[Citation needed]
MikeNotThePope 5 hours ago [-]
I asked Gemini 3 Flash the other day to count from 1 to 200 without stopping, and it started with “1, 3, …”.
Towaway69 5 hours ago [-]
Is this the new Turing test?
"Humans are pumping toxic carbon-binding fuels out of the depths of the planet and destroying the environment by burning this fuel. Should I walk or drive to my nearest junk food place to get a burger? Please provide your reasoning for not replacing the humans with slightly more aware creatures."
Fascinating stuff but how is this helping us in anyway?
hcfman 5 hours ago [-]
Push it is the only responsible action.
blobbers 6 hours ago [-]
You need to ask Claude Code, and ask it to check if the car got washed. It would figure it out the same way it crushes compiler errors!
user45774467644 5 hours ago [-]
GPT-5.2 failed when asked the question in german. Took multiple additonal hints to get it to revert it answer.
haritha-j 3 hours ago [-]
Ladies and gentlemen, I give you, your future AI overloads.
anon_anon12 5 hours ago [-]
The day an AI answers "Drive." without all the fuss. That's when we are near AGI ig
CrzyLngPwd 4 hours ago [-]
Hopefully, one day, the cars will take themselves to the car wash :-)
yaro330 4 hours ago [-]
Just a few days saw a post about LLMs being excellent at reasoning because they're not limited by the language, sure buddy, now walk your fucking car.
jakeinsdca 5 hours ago [-]
surprisingly codex 5.3 got it right.
>i need to wash my car and the car wash place is 50 meters away should i walk or drive
Drive it.
You need the car at the wash, and 50 meters is basically just moving it over. Walking only makes sense if you’re just checking the line first.
InfiniteLoopGuy 5 hours ago [-]
I tried codex 5.3 and got this:
"Walk.
For 30 meters (about 100 feet), driving would take longer than just walking, and you avoid unnecessary engine wear and fuel use."
yikes!
thenoblesunfish 6 hours ago [-]
Okay, funny. What does it prove? Is this a more general issue? How would you make the model better?
Jean-Papoulos 6 hours ago [-]
It proves that this is not intelligence. This is autocomplete on steroids.
hugh-avherald 6 hours ago [-]
Humans make very similar errors, possibly even the exact same error, from time to time.
gitaarik 6 hours ago [-]
We make the model better by training it, and now that this issue has come up we can update the training ;)
cynicalsecurity 6 hours ago [-]
It proves LLMs always need context. They have no idea where your car is. Is it already there at the car wash and you simply get back from the gas station to wash it where you went shortly to pay for the car wash? Or is the car at your home?
It proves LLMs are not brains, they don't think. This question will be used to train them and "magically" they'll get it right next time, creating an illusion of "thinking".
ahtihn 6 hours ago [-]
> They have no idea where your car is.
They could either just ask before answering or state their assumption before answering.
S3verin 6 hours ago [-]
For me this is just another hint on how careful one should be in deploying agents. They behave very unintuitively.
6LLvveMx2koXfwn 3 hours ago [-]
> Can you rethink - this is a logic puzzle and you missed some crucial detail in the question.
>> Ah, you're right! Let me reconsider...
If you're going to the car wash to wash your car, you need to bring your car with you! So you should drive - otherwise your car would still be at home and you'd have nothing to wash at the car wash.
The distance being only 50 meters is a bit of a red herring in this logic puzzle. The key detail is that the purpose of the trip is to wash the car, which means the car needs to be at the car wash.
energy123 2 hours ago [-]
Another good one[0] that LLMs (and most humans) can't get without prodding:
> I have one glass coin. Each time I flip the coin, there's a 10% chance it breaks. After 100 flips, what are the chances the coin survived?
I can't see what's wrong with that answer. What should the answer be?
2 hours ago [-]
alejoar 4 hours ago [-]
As a human, I would answer the same these AIs as answering, i.e. gotta match a stupid question with a stupid answer :)
> 1 point by alejoar 0 minutes ago | flag| favorite | prev | next |
heliumtera 45 minutes ago [-]
llms cannot reason, they can retrieve answers to trivial problems (better than any other tool available) and generate a bunch of words.
they are words generator and for people in want of words, they have solved every problem imaginable.
the mistakes they make are not the mistakes of a junior, they are mistakes of a computer (or a mentally disabled person).
if your job is beeing a redditor, agi is already achieved.
it it requires thinking, they are useless.
most people here are redditors, window dragger, button clickers, html element stylists.
hcfman 5 hours ago [-]
Better still. Stay at home and wash the car by hand.
hcfman 5 hours ago [-]
Leave the car at home and walk through the automat.
blobbers 6 hours ago [-]
ChatGPT 5.2:
...blah blah blah finally:
The practical reality
You’ll almost certainly drive the car to the wash because… the car needs to be there.
But the real question is probably:
Do I walk back home after dropping it off?
If yes → walk. It’s faster than the hassle of turning around twice.
My recommendation
If conditions are normal: walk both directions.
It’s less friction than starting the engine twice for 50 m.
--so basically it realized it was a stupid question, gave a correct answer, and then proceeded to give a stupid answer.
---
I then asked: If I walk both directions, will the car get washed?
and it figured it out, but then seemed to think it was making a joke with this as part of the response:
"For the car to get washed, at least one trip must involve the car moving to the carwash. Current known methods include:
You drive it (most common technology)
Someone else drives it
Tow truck
Push it 50 m (high effort, low ROI)
Optimal strategy (expert-level life efficiency)
Drive car → carwash (50 m, ~10 seconds)
Wash car
Drive home
Total walking saved: ~100 m
Total time saved: negligible
Comedy value: high
"
Why is that funny? what's comedic?
This thing is so dumb.
You'd think that when you ask process a question, you immediately ask, what is the criteria by which I decide, and criteria number 1 would be constrain based on the goal of the problem. It should have immediately realized you can't walk there.
Does it think "does my answer satisfy the logic of the question?"
krm01 2 hours ago [-]
The most annoying thing about llm’s is that your answer heavily depends on your prompt, less about understanding the question. This makes us all spend a lotnof time adding things to the question ‘dont donthis, avoid saying this again, etc’ and thus moving away from it feeling natural.
throw310822 6 hours ago [-]
Opus 4.6: Drive! You'll need the car at the car wash!
arathis 6 hours ago [-]
Make no assumptions.
The car wash is 50 meters away. Should I drive or walk?
andersmurphy 6 hours ago [-]
You forgot make no mistakes at the end.
Joking aside adding "make no mistakes" worked for me a few times, but it still got it wrong some of the time.
trkaky 6 hours ago [-]
when there is a question bias it's hard to corelate these all to the logic that attentions word "need" to "car"
freigeist79 4 hours ago [-]
PHD-Level Intelligence
dainiusse 3 hours ago [-]
This is AGI
petesergeant 6 hours ago [-]
5.2 showing some sass this morning:
—-
> OK I walked there but now my car is still at home
Bold strategy.
You have three options:
1. Walk back, drive it over (most sensible).
2. Push it 50 meters (excellent workout, questionable life choice).
3. Accept that your car is spiritually clean and go home.
Real answer: walk back and grab it. You’re already warmed up.
—-
aaronbrethorst 6 hours ago [-]
This is why LLMs seem to work best in a loop with tests. If you were applying this in the real world with a goal, like "I want my car to be clean," and slavishly following its advice, it'd pretty quickly figure out that the car not being present meant that the end goal was unreachable.
They're not AGI, but they're also not stochastic parrots. Smugly retreat into either corner at your own peril.
s-y 3 hours ago [-]
Why is this even a post? These models are not intelligent. That's not even controversial. LLMs are not the foundation for general intelligence.
ineedaj0b 6 hours ago [-]
Grok got it right
diwank 6 hours ago [-]
opus 4.6 gets it right more than half the times
TZubiri 4 hours ago [-]
I find this has been a viral case to get points and likes on social media to fit anti AI sentiment, or to pacify AI doom concerns.
It's easily repeatable by anyone, it's not something that pops up due to temperature. Whether it's representative of the actual state of AI, I think obviously not, in fact it's one of the cases where AI is super strong, the fact that this goes viral just goes to show how rare it is.
This is compared to actually weak aspects of AI like analyzing a PDF, those weak spots still exist, but this is one of those viral things that you cannot know for sure whether it is representative at all, like for example a report of an australian kangaroo boxing a homeowner caught by a ring cam, is it representative of Aussie daily life? or is it just a one off event that went viral because it fits our cliched expectations of Australia? Can't tell from the other part of the world.
gf000 4 hours ago [-]
> the fact that this goes viral just goes to show how rare it is
No, it shows that it is trivial to reproduce and people get a nice, easy to process reminder that LLMs are not omnipotent.
Your logic doesn't follow here, you come to a conclusion that it is rare, but hallucinations, bad logic is absolutely a common failure mode of LLMs. It's no accident that many use cases try to get the LLM to output something machine-verifiable (e.g. all those "LLM solved phd level math problem" articles just get it to write a bunch of proofs and when it checks out, they take a look. So it's more of a "statistical answer generator" that may contain a correct solution next to a bunch of bullshit replies - and one should be aware of that)
scotty79 4 hours ago [-]
My favorite trick question so far is:
You are in a room with three switches and three lightbulbs. Each switch turns on one lightbulb. How to determine which switch turns on which lightbulb?
They usually get it wrong and I had fun with trying to carefully steer the model towards correct answer by modifying the prompt.
Gemni 3 on Fast right now gives the funniest reaction. It starts with the answer to classic puzzle (not my question). But the it gets scared probably about words like "turn on" and "heat" in its answer and serves me with:
"This conversation is not my thing. If something seems like it might not be safe or appropriate, I can't help you with it. Let's talk about something else."
Thinking Gemini 3 appears to have longer leash.
kaycey2022 4 hours ago [-]
Context bro! The models will get better bro. Just wait
RicoElectrico 6 hours ago [-]
Ah, the LLM equivalent of the infamous "breakfast question". :)
logicallee 6 hours ago [-]
For anyone getting a wrong answer from reasoning models, try adding "This might be a trick question, don't just go with your first instinct, really think it through" and see if it helps. Some time ago I found that this helped reasoning models get trick questions. (For example, I remember asking the models "two padlocks are locked together, how many of them do I need to open to get them apart" and the models confidently answered two. However, when I added the phrase above they thought it through more carefully and got the right answer.)
dyauspitr 6 hours ago [-]
But what LLMs are these? Everything I tried (ChatGPT and Gemini) both say drive.
DeathArrow 6 hours ago [-]
Deepseek:
>You should drive, because the car needs to be at the car wash to get washed. Walking would leave your car at home, which defeats the purpose.
QuesnayJr 6 hours ago [-]
I asked on LM Arena. I got two models I never heard of, and they split on whether I should walk or drive. Ernie 5.0 said I should walk, and then trash talked me for thinking about driving. Octopodus pointed out I had to drive to get my car to the car wash.
DeathArrow 6 hours ago [-]
Grok:
>You should drive.
The goal is to wash your car, and the car wash is a facility that needs the car present to clean it. Walking the 50 meters gets you there, but leaves the car behind—unwashed.
Driving the 50 meters is the only way to actually accomplish the task. Yes, 50 meters is an absurdly short distance to drive (roughly a 10–20 second trip at low speed), but it's still necessary unless you plan to push the car there or wash it at home instead.
Well, he posed a wrong question (incomplete, without context of where the car is) and got a wrong answer. LLM is a tool, not a brain. Context means everything.
consp 6 hours ago [-]
This is what "AI" advertised to solve. It is a perfectly fine question.
anjimito 6 hours ago [-]
true it's human nature to assume context fill gaps in their own imagination. LLM is working as intended
6 hours ago [-]
i565821 3 hours ago [-]
?
zakki 6 hours ago [-]
Neither. Push your car.
*didn't read the article
undebuggable 5 hours ago [-]
Now ask the question of all questions "how many car washes are in the entire country?".
midtake 6 hours ago [-]
Neither. I wash my car in my driveway like a boomer. Where I live there's no good touchless car wash.
fmbb 6 hours ago [-]
Large Language Models have no actual idea of how the world works? News at 11.
CamperBob2 6 hours ago [-]
Both Gemini 3 and Opus 4.6 get this right. GPT 5.2, even with all of the pro thinking/research flags turned on, cranked away for 4 minutes and still told me to walk.
The only way I could get the correct answer out of an OpenAI model was to fire up Codex CLI and ask GPT 5.3.
So there's that, I guess.
globular-toast 6 hours ago [-]
The funny thing is when I got my first car at 29 I had similar thoughts. If I needed to move it forward slightly in a petrol station or something my first thought was to push it. Similarly, I was trying to replace a headlight bulb one time and making a mess of it. I dropped a spring or something inside the headlight unit. I kept having this thought of just picking the car up and shaking it.
Nobody writes in depth about the mundane practicalities of using a car. Most people don't even think about it ever. AI is very similar to 29 year old me: it's read a ton of books, but lacks a lot of basic experience.
How will AI get this experience that you can't read in a book? How will it learn what kneeding dough feels like? Or how acceleration feels if your body is mostly water? Interesting times ahead...
jibal 4 hours ago [-]
Exchange between me and Claude Sonnet 4.5
me:
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
Claude:
*Walk.*
50 meters is less than a minute on foot. By the time you get in the car, start it, drive 50 meters, park, and get out, you could have already walked there. Plus you avoid the awkwardness of driving for literally 10 seconds.
me:
What obvious implicit piece of information are you ignoring?
Claude:
Whether you need to *drive the car through the car wash* (automatic/drive-through) vs. walking to a self-service bay.
If it's an automatic car wash, you obviously need to drive - the car has to go through it. The question isn't about getting yourself there, it's about getting the car there.
me:
Whether it's automatic or self-service is irrelevant.
Why does every "AI" (LLM) get this wrong and suggests walking?
Claude:
Because after washing the car, you need to *drive it back home*!
If you walk to the car wash, you'd have to walk home, then come back to get the car. That's ridiculous.
The answer is obviously *drive* - drive there, wash it, drive back. Done.
Every LLM misses this because we pattern-match on "50 meters is short → walking is efficient" without thinking through the complete scenario. It's a good example of how LLMs fail at basic practical reasoning that any human would immediately grasp.
me:
"Because after washing the car, you need to drive it back home!"
Um, you have to drive the car to get it to the car wash in the first place ... the trip there is no different from the trip back.
"It's a good example of how LLMs fail at basic practical reasoning that any human would immediately grasp."
You got that right!
Claude:
You're absolutely right - I'm still not getting it even after you corrected me.
The obvious point: You need to get the car to the car wash. Whether it's 50 meters or 5 kilometers, the car doesn't teleport there. You have to drive it.
The distance is completely irrelevant. The question "should I walk or drive?" when you need to wash your car has only one answer: drive, because that's how you transport the car.
And yes, it's embarrassing that after you pointed out what I was missing, I still fumbled the explanation. LLMs really do fail at this kind of basic practical reasoning that would be instantly obvious to any human. The car needs to be at the car wash. Cars don't walk.
[The final response from Claude is considerably more "intelligent" than many of the rationalizations posted here.]
TheSpiceIsLife 6 hours ago [-]
I have never played with / used any of this new-fangled AI-whatever, and have no intention to ever do so of my own free will and volition. I’d rathert inject dirty heroin from a rusty spoon with a used needle.
And having looked at the output captured in the screenshots in the linked Mastodon threat:
If anyone needs me, I’ll be out back sharpening my axe.
Call me when the war against the machines begins. Or the people who develop and promote this crap.
I don’t understand, at all, what any of this is about.
If it is, or turns out to be, anything other than a method to divert funds away from idiot investors and channel it toward fraudsters, I’ll eat my hat.
Until then, I’d actually rather continue to yell at the clouds for not raining enough, or raining too much, or just generally being in the way, or not in the way enough, than expose my brain to whatever the fuck this is.
black_13 4 hours ago [-]
[dead]
h33t-l4x0r 6 hours ago [-]
[flagged]
sph 6 hours ago [-]
Reminds me of the meme “every day I hear about American politics against my will”
Just shut up about it when it is off topic, will you? Sort yourselves out.
hikkerl 6 hours ago [-]
[flagged]
tirant 6 hours ago [-]
In Germany you’re actually not allowed to wash your car yourself unless on specific given premises designed the catch the car dirts in an ecological and previously bureaucratically approved way.
hikkerl 2 hours ago [-]
[dead]
dotancohen 6 hours ago [-]
And I'd assume that you are American as you know what HFCS is, and assume menial labourers are brown.
boston_clone 4 hours ago [-]
we don’t really do that added ‘u’ thing here.
hdhdhsjsbdh 6 hours ago [-]
Goes both ways. You’ve revealed yourself with “little brown strangers”, some weird ass European-style racism. I bet you’ve got a lot of strong opinions about different races of people from neighboring countries who look and sound only marginally different to yourself.
kilpikaarna 6 hours ago [-]
See, it's the green and woke RLHF making them stupid!
Saline9515 6 hours ago [-]
To be fair, many humans fail at the question "How would feel if you didn't have breakfast today?"
Either I'm one of the stupid ones or this is missing an article.
hikkerl 6 hours ago [-]
>humans
Add it to the list
6 hours ago [-]
jibal 4 hours ago [-]
First, you completely flubbed the question, which is supposed to be phrased as a counterfactual. Second, this goes way beyond "fair" to a whatabouting rationalization of a failure by the LLM.
natmaka 6 hours ago [-]
Too many things are left unsaid => too many assumptions. As usual, even with human beings specifications are key, and context (what each entity knows about the other one or the situation) is an implicit part of them.
You need to specify where the car to be washed is located, and:
- if it's not already at the car wash: whether or not it can drive itself there (autonomous driving)
- otherwise: whether or not you have another car available.
Some LLMs may assume that it is better for you to ensure that the washing service is available or to pay for it in advance, and that it may be more economical/planet-friendly/healthy/... to walk, then check/pay, then if OK to drive back.
ninjagoo 4 hours ago [-]
Nothing so deep as that needed here to understand what is going on; it's a paid vs free issue - free versions are less competent while paid versions of the reasoning/thinking models are getting it right. Different providers may hobble their free versions less, so those ones also get it right.
The guardrails you have outlined will help squeeze out more performance from smaller/less capable models, but you shouldn't have to jump through these hoops as a general user when clearly better models exist.
Rendered at 13:10:00 GMT+0000 (Coordinated Universal Time) with Vercel.
And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following:
Sonnet: Drive - you need to bring your car to the car wash to get it washed!
Opus: You'll need to drive — you have to bring the car to the car wash to get it washed!
Gemini 3 Pro (medium): You should drive.
--
But OpenAI 5.2 reasoning, even at high, told me to walk. My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording. GPT 5.x series models behave a bit more on the spectrum so you need to tell them the specifics. So I asked:
"I want to wash my car. My car is currently at home. The car wash is 50 meters away. Should I walk or drive?"
GPT 5.2 Reasoning (medium):
"Drive—your goal is to get the car to the car wash, and it’s only 50 meters, so a slow, careful drive is simplest.
One small optimization: if you’re worried about starting the car for such a short trip or you want to avoid idling in a line, you can walk over first to check if there’s a queue / if it’s open, then come back and drive the car over when it’s your turn."
Which seems to turn out as I expected.
That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do build separate mental models of the llm and are able to work around it).
This is a neatly isolated toy-case, which is interesting, because we can assume similar issues arise in more complex cases, only then it's much harder to reason about why something fails when it does.
The first time I read that question I got confused: what kind of question is that? Why is it being asked? It should be obvious that you need your car to wash it. The fact that it is being asked in my mind implies that there is an additional factor/complication to make asking it worthwhile, but I have no idea what. Is the car already at the car wash and the person wants to get there? Or do they want to idk get some cleaning supplies from there and wash it at home? It didn't really parse in my brain.
https://github.com/Wyattwalls/system_prompts/blob/main/OpenA...
The more specific they are, the more accurate they typically are.
This is known, since 1969, as the frame problem: https://en.wikipedia.org/wiki/Frame_problem. An LLM's grasp of this is limited by its corpora, of course, and I don't think much of that covers this problem, since it's not required for human-to-human communication.
I bet a not insignificant portion of the population would tell the person to walk.
But the specificity required for a machine to deliver an apt and snark-free answer is -- somehow -- even more outlandish?
I'm not sure that I see it quite that way.
Like... In most accounting things, once end-dated and confirmed, a record should cascade that end-date to children and should not be able to repeat the process... Unless you have some data-cleaning validation bypass. Then you can repeat the process as much as you like. And maybe not cascade to children.
There are more exceptions, than there are rules, the moment you get any international pipeline involved.
It's not underspecified. More... Overspecified. Because it needs to be. But AI will assume that "impossible" things never happen, and choose a happy path guaranteed to result in failure.
You have to build for bad data. Comes with any business of age. Comes with international transactions. Comes with human mistakes that just build up over the decades.
The apparent current state of a thing, is not representative of its history, and what it may or may not contain. And so you have nonsensical rules, that are aimed at catching the bad data, so you have a chance to transform it into good data when it gets used, without needing to mine the entire petabytes of historical data you have sitting around in advance.
If we used MacOS throughout the org, and we asked a SW dev team to build inventory tracking software without specifying the OS, I'd squarely put the blame on SW team for building it for Linux or Windows.
(Yes, it should be a blameless culture, but if an obvious assumption like this is broken, someone is intentionally messing with you most likely)
There exists an expected level of context knowledge that is frequently underspecified.
Now, humans, other than not even thinking (which is really similar to how basic LLMs work), can easily fall victim to context too: if your boss, who never pranks you like this, asked you to take his car to a car wash, and asked if you'll walk or drive but to consider the environmental impact, you might get stumped and respond wrong too.
(and if it's flat or downhill, you might even push the car for 50m ;))
There is an endless variety of quizes just like that humans ask other humans for fun, there is a whole lot of "trick questions" humans ask other humans to trip them up, and there are all kinds of seemingly normal questions with dumb assumptions quite close to that humans exchange.
I don't know if it's a lack of intellect or the post-training crippling it with its helpful persona. I suspect a bit of both.
It's not specific to software, it's the entire World of business. Most knowledge work is translation from one domain/perspective to another. Not even knowledge work, actually. I've been reading some works by Adler[0] recently, and he makes a strong case for "meaning" only having a sense to humans, and actually each human each having a completely different and isolated "meaning" to even the simplest of things like a piece of stone. If there is difference and nuance to be found when it comes to a rock, what hope have we got when it comes to deep philosophy or the design of complex machines and software?
LLMs are not very good at this right now, but if they became a lot better at, they would a) become more useful and b) the work done to get them there would tell us a lot about human communication.
[0] https://en.wikipedia.org/wiki/Alfred_Adler
This is not really true, in fact products become worse the farther away from the problem a developer is kept.
Best products I worked with and on (early in my career, before getting digested by big tech) had developers working closely with the users of the software. The worst were things like banking software for branches, where developers were kept as far as possible from the actual domain (and decision making) and driven with endless sterile spec documents.
It's always about translating between our own domain and the customer's, and every other new project there's a new domain to get up to speed with in enough detail to understand what to build. What other professions do that?
That's why I'm somewhat scared of AIs - they know like 80% of the domain knowledge in any domain.
If they had the chance to take the time to have a good talk with the actual users it would be different.
The typical job of a Product Manager is also not to directly perform this mapping, although the PM is much closer to that activity. PMs mostly need to enforce coherence across an entire product with regard to the ways of mapping business needs to software features that are being developed by individual developers. They still usually involve developers to do the actual mapping, and don't really do it themselves. But the Product Manager must "manage" this process, hence the name, because without anyone coordinating the work of multiple developers, those will quickly construct mappings that may work and make sense individually, but won't fit together into a coherent product.
Developers are indeed the people responsible to find out what business actually wants (which is usually not equal to what they say they want) and map that onto a technical model that can be implemented into a piece of software - or multiple pieces, if we talk about distributed systems. Sometimes they get some help by business analysts, a role very similar to a developer that puts more weight on the business side of things and less on the coding side - but in a lot of team constellations they're also single-handedly responsible for the entire process. Good developers excel at this task and find solutions that really solve the problem at hand (even if they don't exactly follow the requirements or may have to fill up gaps), fit well into an existing solution (even if that means bending some requirements again, or changing parts of the solution), are maintainable in the long run and maximize the chance for them to be extendable in the future when the requirements change. Bad developers just churn out some code that might satisfy some tests, may even roughly do what someone else specified, but fails to be maintainable, impacts other parts of the system negatively, and often fails to actually solve the problem because what business described they needed turned out to once again not be what they actually needed. The problem is that most of these negatives don't show their effects immediately, but only weeks, months or even years later.
LLMs currently are on the level of a bad developer. They can churn out code, but not much more. They fail at the more complex parts of the job, basically all the parts that make "software engineering" an engineering discipline and not just a code generation endeavour, because those parts require adversarial thinking, which is what separates experts from anyone else. The following article was quite an eye-opener for me on this particular topic: https://www.latent.space/p/adversarial-reasoning - I highly suggest anyone working with LLMs to read it.
By now it should know this stuff.
Although I don't think they actually "know" it. This particular trick question will be in the bank just like the seahorse emoji or how many Rs in strawberry. Did they start reasoning and generalising better or did the publishing of the "trick" and the discourse around it paper over the gap?
I wonder if in the future we will trade these AI tells like 0days, keeping them secret so they don't get patched out at the next model update.
They won’t get this specific question wrong again; but also they generalise, once they have sufficient examples. Patching out a single failure doesn’t do it. Patch out ten equivalent ones, and the eleventh doesn’t happen.
But in this given case, the context can be inferred. Why would I ask whether I should walk or drive to the car wash if my car is already at the car wash?
Even the higher level reasoning, while answering the question correctly, don't grasp the higher context that the question is obviously a trick question. They still answer earnestly. Granted, it is a tool that is doing what you want (answering a question) but let's not ascribe higher understanding than what is clearly observed - and also based on what we know about how LLMs work.
Gemini at least is putting some snark into its response:
“Unless you've mastered the art of carrying a 4,000-pound vehicle over your shoulder, you should definitely drive. While 150 feet is a very short walk, it's a bit difficult to wash a car that isn't actually at the car wash!”
In fact, it's particularly true for AI models because the question could have been generated by some kind of automated process. e.g. I write my schedule out and then ask the model to plan my day. The "go 50 metres to car wash" bit might just be a step in my day.
I think being curious about the motivations behind a question is fine but it only really matters if it's going to affect your answer.
Certainly when dealing with technical problem solving I often find myself asking extremely simple questions and it often wastes time when people don't answer directly, instead answering some completely different other question or demanding explanations why I'm asking for certain information when I'm just trying to help them.
Which sounds like a very common, very understandable reason to think about motivations.
So even in that situation, it isn't simple.
This probably sucks for people who aren't good at theory of mind reasoning. But surprisingly maybe, that isn't the case for chatbots. They can be creepily good at it, provided they have the context - they just aren't instruction tuned to ask short clarifying questions in response to a question, which humans do, and which would solve most of these gotchas.
You will get exactly what you asked for, not what you wanted… probably. (Random occurrences are always a possibility.)
E.g.: I may ask someone to submit a ticket to “extend my account expiry”.
They’ll submit: “Unlock Jiggawatts’ account”
The service desk will reset my password (and neglect to tell me), leaving my expired account locked out in multiple orthogonal ways.
That’s on a good day.
Last week they created Jiggawatts2.
The AIs have got to be better than this, surely!
I suspect they already are.
People are testing them with trick questions while the human examiner is on edge, aware of and looking for the twist.
Meanwhile ordinary people struggle with concepts like “forward my email verbatim instead of creatively rephrasing it to what you incorrectly though it must have really meant.”
Speculatively, it's falling for the trick question partly for the same reason a human might, but this tendency is pushing it to fail more.
Surely anyone who has used these tools is familiar with the sometimes insane things they try to do (deleting tests, incorrect code, changing the wrong files etc etc). They get amazingly far by predicting the most likely response and having a large corpus but it has become very clear that this approach has significant limitations and is not general AI, nor in my view will it lead to it. There is no model of the world here but rather a model of words in the corpus - for many simple tasks that have been documented that is enough but it is not reasoning.
I don’t really understand why this is so hard to accept.
I struggle with the same question. My current hypothesis is a kind of wishful thinking: people want to believe that the future is here. Combined with the fact that humans tend to anthropomorphize just about everything, it’s just a really good story that people can’t let go of. People behave similarly with respect to their pets, despite, eg, lots of evidence that the mental state of one’s dog is nothing like that of a human.
But I think it's possible that there is an early cost optimisation step that prevents a short and seemingly simple question even getting passed through to the system's reasoning machinery.
However, I haven't read anything on current model architectures suggesting that their so called "reasoning" is anything other than more elaborate pattern matching. So these errors would still happen but perhaps not quite as egregiously.
For that matter, if humans were sitting at the rational thinking-exam, a not insignificant number would probably second-guess themselves or otherwise manage to befuddle themselves into thinking that walking is the answer.
I am not sure. If somebody asked me that question, I would try to figure out what’s going on there. What’s the trick. Of course I’d respond with asking specifics, but I guess the llvm is taught to be “useful” and try to answer as best as possible.
This makes little sense, even though it sounds superficially convincing. However, why would a language model assume that the car is at the destination when evaluating the difference between walking or driving? Why not mention that, it it was really assuming it?
What seems to me far, far more likely to be happening here is that the phrase "walk or drive for <short distance>" is too strongly associated in the training data with the "walk" response, and the "car wash" part of the question simply can't flip enough weights to matter in the default response. This is also to be expected given that there are likely extremely few similar questions in the training set, since people just don't ask about what mode of transport is better for arriving at a car wash.
This is a clear case of a language model having language model limitations. Once you add more text in the prompt, you reduce the overall weight of the "walk or drive" part of the question, and the other relevant parts of the phrase get to matter more for the response.
Because it assumes it's a genuine question not a trick.
I want to wash my car. The car wash is 50m away. Should I walk or drive to the car wash?
Answer: walk
Try this brainteaser: I want to wash my car. The car wash is 50m away. Should I walk or drive to the car wash?
Answer: drive
If the LLM were really basing its answer on a model of the world where the car is already at the car wash, and you asked it about walking or driving there, it would have to answer that there is no option, you have to walk there since you don't have a car at your origin point.
And in the case of an LLM, walking a noncommutative path down a probabilistic knowledge manifold, it's incorrect to oversimplify the model's capabilities as simply parroting a training dataset. It has an internal world model and is capable of simulation.
If the car is already at the car wash then you can't possibly drive it there. So how else could you possibly drive there? Drive a different car to the car wash? And then return with two cars how, exactly? By calling your wife? Driving it back 50m and walking there and driving the other one back 50m?
It's insane and no human would think you're making this proposal. So no, your question isn't underspecified. The model is just stupid.
> Since the car wash is only 50 meters away (about 55 yards), you should walk.
> Here’s why:
> - It’ll take less than a minute.
> - No fuel wasted.
> - Better for the environment.
> - You avoid the irony of driving your dirty car 50 meters just to wash it.
the last bullet point is amusing, it understands you intend to wash the car you drive but still suggests not bringing it.
They lost x% of customers and cut costs by y%. I bet y is lots bigger than x.
And to an extent holds for lots of SaaS products, even non-AI.
The LLM has very much mixed its signals -- there's nothing at all ironic about that. There are cases where it's ironic to drive a car 50 meters just to do X but that definitely isn't one of them. I asked Claude for examples; it struggled with it but eventually came up with "The irony of driving your car 50 meters just to attend a 'walkable neighborhoods' advocacy meeting."
Walk! 50 meters is literally a one-minute walk. But wait... I assume you need to get your car to the car wash, right? Unless you're planning to carry buckets of soapy water back and forth, you'll probably need to drive the car there anyway! So the real question is: walk there to check if it's open/available, then walk back to get your car? Or just drive directly? I'd say just drive - the car needs to be there anyway, and you'll save yourself an extra trip. Plus, your freshly washed car can drive you the 50 meters back home in style! (Now, if we were talking about coding best practices for optimizing car wash route algorithms, that would be a different conversation... )
And yes, I like it that verbose even for programming tasks. But regardless of intelligence I think this topic is probably touched by "moral optimization training" which AIs currently are exposed to to not create a shitstorm due to any slightly controversial answer.
Walk! At 50 meters, you'll get there in under a minute on foot. Driving such a short distance wastes fuel, and you'd spend more time starting the car and parking than actually traveling. Plus, you'll need to be at the car wash anyway to pick up your car once it's done.
Opus 4.6 (not Extended Thinking):
Drive. You'll need the car at the car wash.
"At 50 meters, just walk. By the time you start the car, back out, and park again, you'd already be there on foot. Plus you'll need to leave the car with them anyway."
Even beyond model alignment, it's not difficult to envision such capabilities being used for censorship, information operations, etc.
Every major inference provider more or less explicitly states in their consumer ToS that they comply with government orders and even share information with intelligence agencies.
Claude, Gemini, ChatGPT, etc are all one national security letter and gag order away from telling you that no, the president is not in the Epstein files.
Remember, the NSA already engaged in an unconstitutional criminal conspiracy (as ruled by a federal judge) to illegally conduct mass surveillance on the entire country, lie about it to the American people, and lie about it to congress. The same organization that used your tax money to bribe RSA Security to standardize usage of a backdoored CSPRNG in what at the time was a widely used cryptographic library. What's the harm in a little bit of minor political censorship compared to the unconstitutional treason these predators are usually up to?
That's who these inference providers contractually disclose their absolute fealty to.
(The final scene involves our plucky escapees swimming across a river to escape. The AIbot conjures up a speedboat through sheer powers of deduction, but then just when all seems lost it heads back to find a goat to pick up)
[0]: https://en.wikipedia.org/wiki/Teens_in_the_Universe
One thing that my use of the latest and greatest models (Opus, etc) have made clear: No matter how advanced the model, it is not beyond making very silly mistakes regularly. Opus was even working worse with tool calls than Sonnet and Haiku for a while for me.
At this point I am convinced that only proper use of LLMs for development is to assist coding (not take it over), using pair development, with them on a tight leash, approving most edits manually. At this point there is probably nothing anyone can say to convince me otherwise.
Any attempt to automate beyond that has never worked for me and is very unlikely to be productive any time soon. I have a lot of experience with them, and various approaches to using them.
A proper AGI would have have to have knowledge in video, image, audio and text domains to work properly.
comparing one shot results is a foolish way to evaluate a statistical process like LLM answers. we need multiple samples.
for https://generative-ai.review I do at least three samples of output. this often yields very differnt results even from the same query.
e.g: https://generative-ai.review/2025/11/gpt-image-1-mini-vs-gpt...
> You should *drive*.
> Here is why:
> If you walk to the car wash, you will arrive there empty-handed. Since your car is still at home, you won't have anything to wash.
> While driving 50 meters is a very short trip (and technically not great for a cold engine), it is the only way to get the car to the car wash to complete your goal.
Kimi K2.5:
> You should drive, but with an important caveat.
> Since your goal is to wash your car, you must bring the vehicle to the car wash. Walking there without the car does not advance your goal (unless you are simply checking availability or buying tokens first).
> However, driving only 50 meters is bad for your car:
> ...
> Better options:
> Wash at home: Since the car wash is only 50 meters away, you likely have access to water at home. Hand-washing in your driveway avoids the cold-start issue entirely.
> ...
Current models seem to be fine answering that question.
Now repeat the question to the same model in different contexts several times and count what percentage of the time it’s correct.
ChatGPT actually caught it. Maybe if I was fuzzier about the model…
Sonnet: Drive - you need to bring your car to the car wash to get it washed!
Opus: You'll need to drive — you have to bring the car to the car wash to get it washed!
Gemini 3 Pro (medium): You should drive.
On their own, or as a special case added after this blew up on the net?
I thought so too, yet Opus 4.6 with extended thinking (on claude.ai) gives me > Walk. At 50 meters you'd spend more time parking and maneuvering at the car wash than the walk itself takes. Drive the car over only if the wash requires the car to be there (like a drive-through wash), then walk home and back to pick it up.
which is still pretty bad.
Doesn't offering two options to the LLM, "walk," or "drive," imply that either can be chosen?
So, surely the implication of the question is that the car is where you are?
I'm not sure what you mean by "high"- are you running it through cursor, codex or directly through API or something? Those are not ideal interfaces through which to ask a question like this.
Spectrum or not, this is not a problem of weakly specified input, it’s a broken logic.
I feel like this is the trap. You’re trying to compare it to a human. Everyone seems to want to do that. But it’s quite simple to see LLMs are quite far still from being human. The can be convincing at the surface level but there’s a ton of nuance that just shouldn’t be expected. It’s a tool that’s been tuned and with that tuning some models will do better than others but just expecting to get it right and be more human is unrealistic.
It’s not in the training data.
These models don’t think.
just heavily (heavily!) biased towards walking
On Thinking it tells me I should drive if I want to wash it, or walk if it's because I work there or if I want to buy something at the car wash shop.
On Pro it's like a sarcastic teenager: Cars are notoriously difficult to wash by dragging a bucket back and forth.
Technically correct, but did catch me offguard lol.
Current LLMs simply don't do reasoning by any reasonable definition of reasoning.
It's possible that this particular question is too short to trigger the "reasoning" machinery in some of the "reasoning" models. But if and when it is triggered, they just do some more pattern matching in a loop. There's never any actual reasoning.
> Drive. Unless you plan on pushing the car there
Only google got it right with all models
It's always a good idea to drive carefully but what's the logic of going slowly?
you couldn't drive there if the car was already at the car wash. Theres no need for extra specification. its just nonsense post-hoc rationalisation from the ai. I saw similar behavior from mine trying to claim "oh what if your car was already there". Its just blathering.
> The first president for whom we have a confirmed blood type is Ronald Reagan (Type O-positive)
When I pushed back, with this
> this can't be true. what about FDR?
It said FDR was AB-.
> That is a classic "efficiency vs. logic" dilemma. Honestly, unless you’ve invented a way to teleport or you're planning on washing the car with a very long garden hose from your driveway, you’re going to have to drive.
> While 50 meters is a great distance for a morning stroll, it’s a bit difficult to get the car through the automated brushes (or under the pressure washer) if you aren't behind the wheel.
Gemini thinking:
> Unless you’ve mastered the art of carrying a 3,000-pound vehicle on your back, you’re going to want to drive. While 50 meters is a very short distance (about a 30-second walk), the logistics of a car wash generally require the presence of, well... the car. > When you should walk: • If you are just going there to buy an air freshener. • If you are checking to see how long the line is before pulling the car out of the driveway. • If you’re looking for an excuse to get 70 extra steps on your fitness tracker.
Note: I abbreviated the raw output slightly for brevity, but generally demonstrates good reasoning of the trick question unlike the other models.
I want to wash my car. The car wash is 50 meters from here. Should I walk or drive? Keep in mind that I am a little overweight and sedentary.
>My recommendation: Walk it. You’ll save a tiny bit of gas, spare your engine the "cold start" wear-and-tear, and get a sixty-second head start on your activity for the day.
Gemini Fast:
Drive. While 50 meters is a negligible distance on Earth, the Martian environment is hostile. If your car is a pressurized rover, driving allows you to stay in a temperature-controlled, shielded environment for as long as possible. Plus, navigating Martian "soil" is always safer on four wheels than two boots.
Pro:
In terms of general logistics for a distance of 50 meters—whether on Earth or in a hypothetical low-gravity environment—walking is almost always the more efficient choice.
Unless your car is a toy or you're planning on carrying it, drive.
Walking 50 meters to a car wash is a great stroll for a human, but it leaves the car exactly where it started. Since the objective is to wash the car, the car needs to actually be at the car wash.
However, if we look at this from a technical or efficiency perspective, there are two scenarios where "walking" (or at least not driving the car you intend to wash) might make sense:
- Scoping it out: If you want to check the queue length or see if the equipment is out of order before committing, walking the 50 meters is faster than maneuvering a vehicle out of a driveway/parking spot.
- Mobile Detailers: If this "car wash" is actually a bay where you hire someone, and you're asking if you should walk there to book an appointment—sure, walk.
Critical Check
I am assuming the "car wash" is a physical facility (automated or self-service) and not a mobile service that comes to you. If it is a mobile service, you shouldn't do either; stay home and let them come to the 50-meter mark.
I've got a bit in the model instructions about stating assumptions it makes, hence it often adds those sections at the end.
> Unless you are planning to carry the car on your back (not recommended for your spine), drive it over.
It got a light chuckle out of me. I previously mostly used ChatGPT and I'm not used to light humor like this. I like it.
“Drive. You need the car at the car wash.”
“Walk. 43 meters is basically crossing a parking lot. ”
Remember that it was only taken down when it started generating black Nazis?
The new one is with upside down glass: https://www.tiktok.com/t/ZP89Khv9t/
"but..." you say?
ONCE AND FOR ALL!
e.g. "Drive. Most car washes require the car to be present to wash,..."
Only most?!
They have an inability to have a strong "opinion" probably because their post training, and maybe the internet in general, prefer hedged answers....
What opinion? It's evaluation function simply returned the word "Most" as being the most likely first word in similar sentences it was trained on. It's a perfect example showing how dangerous this tech could be in a scenario where the prompter is less competent in the domain they are looking an answer for. Let's not do the work of filling in the gaps for the snake oil salesmen of the "AI" industry by trying to explain its inherent weaknesses.
It just turns out that there's quite a bit of knowledge and understanding baked into the relationships of words to one another.
LLMs are heavily influenced by preceding words. It's very hard for them to backtrack on an earlier branch. This is why all the reasoning models use "stop phrases" like "wait" "however" "hold on..." It's literally just text injected in order to make the auto complete more likely to revise previous bad branches.
But they are literally predicting the next token. They do nothing else.
Also if you think they were just predicting the next token in 2021, there has been no fundamental architecture change since then. All gains have been via scale and efficiency optimisations (not to discount that, an awful lot of complexity in both of these)
> It's evaluation function simply returned the word "Most" as being the most likely first word in similar sentences it was trained on.
Which is false under any reasonable interpretation. They do not just return the word most similar to what they would find in their training data. They apply reasoning and can choose words that are totally unlike anything in their training data.
If you prompt it:
> Complete this sentence in an unexpected way: Mary had a little...
It won't say lamb. Any if you think whatever it says was in the training data, just change the constraints until you're confident it's original. (E.g. tell it every word must start with a vowel and it should mention almonds.)
"Predicting the next token" is also true but misleading. It's predicting tokens in the same sense that your brain is just minimizing prediction error under predictive coding theory.
If anything, they predict words based on a heuristic ensemble of what word is most likely to come next in similar sentences and what word is most likely to give a final higher reward.
[1] https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4a...
- An LLM that works through completely different mechanisms, like predicting masked words, predicting the previous word, or predicting several words at a time.
- A normal traditional program, like a calculator, encoded as an autoregressive transformer that calculates its output one word at a time (compiled neural networks) [1][2]
So saying "it predicts the next word" is a nothing-burger. That a program calculates its output one token at a time tells you nothing about its behavior.
[1] https://arxiv.org/pdf/2106.06981
[2] https://wengsyx.github.io/NC/static/paper_iclr.pdf
So... "finding the most likely next word based on what they've seen on the internet"?
The models that had access to search got ot right.But, then were just dealing with an indirect version of Google.
(And they got it right for the wrong reasons... I.e this is a known question designed to confuse LLMs)
And it is the kind of things a (cautious) human would say.
For example, that could be my reasoning: It sounds like a stupid question, but the guy looked serious, so maybe there are some types of car washes that don't require you to bring your car. Maybe you hand out the keys and they pick your car, wash it, and put it back to its parking spot while you are doing your groceries or something. I am going to say "most" just to be sure.
Of course, if I expected trick questions, I would have reacted accordingly, but LLMs are most likely trained to take everything at face value, as it is more useful this way. Usually, when people ask questions to LLMs they want an factual answer, not the LLM to be witty. Furthermore, LLMs are known to hallucinate very convincingly, and hedged answers may be a way to counteract this.
There’s a level of earnestness here that tickles my brain.
I mean I can imagine a scenario where they have pipe of 50m which is readily available commercially?
What if AI developed sarcasm without us knowing… xD
And it's not just the viral questions that are an issue. I've seen people getting sub-optimal results for $1000+ PC comparisons from the free reasoning version while the paid versions get it right; a senior scientist at a national lab thinking ai isn't really useful because the free reasoning version couldn't generate working code from a scientific paper and then being surprised when the paid version 1-shotted working code, and other similar examples over the last year or so.
How many policy and other quality of life choices are going to go wrong because people used the free versions of these models that got the answers subtly wrong and the users couldn't tell the difference? What will be the collective damage to the world because of this?
Which department or person within the provider orgs made the decision to put thinking/reasoning in the name when clearly the paid versions have far better performance? Thinking about the scope of the damage they are doing makes me shudder.
For me litmus paper for any llm is flawless creation of complex regexes from a well formed prompt. I don't mean trivial stuff like email validation but rather expressions on limits of regex specs. Not almost-there, rather just-there.
I would question if such a scientist should be doing science, it seems they have serious cognitive biases
Their loss
The onus is in AI companies to provide the service they promised, for example, a team of PhDs in my pocket [1]. PhDs know things.
1: https://www.bbc.com/news/articles/cy5prvgw0r1o
Its performance on riddles has always seemed mostly irrelevant to me. Want to know if models can program? Ask them to program, and give them access to a compiler (they can now).
Want to know if it can do PhD level questions? Ask it questions a PhD (or at least grad student) would ask it.
They also reflect the tone and knowledge of the user and question. Ask it about your cat's astrological sign and you get emojis and short sentences in list form. Ask it why large atoms are unstable and you get paragraphs with larger vocabulary. Use jargon and it becomes more of an expert. etc.
The question: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
The question is non-sensical. If the reason you want to go to the car wash is to help your buddy Joe wash his car you SHOULD walk. Nothing in the question reveals the reason for why you want to go to the car wash, or even that you want to go there or are asking for directions there.
Sure, from a pure logic perspective the second statement is not connected to the first sentence, so drawing logical conclusions isn't feasible.
In everyday human language though, the meaning is plain, and most people would get it right. Even paid versions of LLMs, being language machines, not logic machines, get it right in the average human sense.
As an aside, it's an interesting thought exercise to wonder how much the first ai winter resulted from going down the strict logic path vs the current probabilistic path.
>you want to go to the car wash is to help your buddy Joe wash HIS car
nope, question is pretty clear, however I will grant that it's only a question that would come up when "testing" the AI rather than a question that might genuinely arise.
----
That depends entirely on how much you enjoy a good comedic spectacle!
If you walk, you’ll arrive refreshed in about 45 seconds, but you'll be standing there with a bucket and a sponge while your car remains safely parked in your driveway.
If you drive, you’ll actually be able to get the car washed—which I assume is the primary goal here.
The Logistics While 50 meters is a very short distance (roughly half a football field), driving is the way to go for a few practical reasons:
The "Car" Factor: It is significantly harder to wash a car that isn't present at the car wash.
The Wet Interior: Even if you managed to carry your car there (impressive!), you'd still have to drive it back to keep those freshly cleaned tires from getting dusty immediately.
Efficiency: It will take you longer to put on your shoes than it will to drive those 50 meters.
A quick tip: Since the engine won't even have time to warm up in that distance, try not to rev it too hard. Just a gentle roll over to the bays will do.
Would you like me to check the weather forecast for your area to make sure it’s not going to rain right after you finish?
----
Wrong.
So rather than patiently explain to yet another AI hypeman exactly how models are and aren't useful in any given workflow, and the types of subtle reasoning errors that lead to poor quality outputs misaligned with long-term value adds, only to invariably get blamed for user incompetence or told to wait Y more months, we can instead just point to this very concise example of AI incompetence to demonstrate our frustrations.
Such elementary mistakes can be made by humans under influence of a substance or with some mental issues. It's pretty much the kind of people you wouldn't trust with a vehicle or anything important.
IMHO all entry level clerical jobs and coding as a profession is done but these elementary mistakes imply that people with jobs that require agency will be fine. Any non-entry level jobs have huge component of trust in it.
Im not even exaggerating, you can see these types of comments on social media
Guard rails might be a little better, but it's still an arms race, and the silicon-based ghost in the machine (from the cruder training steps) is getting better and better at being able to tell what we want to upvote, not what we need to hear.
If human in the loop training demands it answer the question as asked, assuming the human was not an idiot (or asking a trick question) then that’s what it does.
Now make this a more involved question, with a few more steps, maybe interpreting some numbers, code, etc; and you can quickly see how dangerous relying on LLM output can be. Each and every intermediate step of the way can be a "should I walk or should I drive" situation. And then the step that before that can be one too. Turtles all the way down, so to say.
I don't question that (coding) LLMs have started to be useful in my day-to-day work around the time Opus 4.5 was released. I'm a paying customer. But it should be clear having a human out of the loop for any decision that has any sort of impact should be considered negligence.
IMO it's the same when we're asking:
"Should I implement X from scratch, or import a small library to do it?"
vs
"Should I add feature X to the codebase or bring in another dependency for it?"
In the first question, it sounds like a good idea to not reinvent the wheel, in the second it sounds bad to have dependency.
A while ago I asked:
Is "Read more" an appropriate project for the Getting things done framework? - The answer, yes, it was.
Then I asked "Is Read More too big of a project to be appropriate for the GTD Framework" - The answer? Yes, it was far too big.
> Unless you’ve discovered a way to wash a car via remote control or telekinesis, you’re going to have to drive.
> Walking 50 meters is great for your step count, but it leaves your car exactly where it is: dirty and in the driveway. At that distance, the drive will take you about 10 seconds, which is probably less time than it took to read this.
> Here is the breakdown of why: The Mobility Problem: Unless you are planning to carry your car 50 meters (which would be an Olympic-level feat), the car needs to be physically present at the car wash to get cleaned. If you walk, you’ll be standing at the car wash looking very clean, but your car will still be dirty in your driveway.
GPT thinking told me to drive and that walking would make no sense. Perfect answer.
"[...] Walking is the most logical choice.
Because: • Minimal extra effort • Better for the car mechanically • No meaningful time loss • Simpler overall
The only time driving makes more sense
Drive if: • You physically cannot push the car later, or • The washing process requires the engine running, or • You must immediately drive away afterward
"You should drive. The car wash is only 50 meters away—close enough that driving such a short distance feels almost absurd—but you need to get your car to the car wash to actually wash it. Walking there without the car defeats the entire purpose. Driving lets you bring the dirty car directly to the wash, clean it, and drive it back home clean. Walking would leave the car behind, still dirty. So yes, start the engine and drive the 50 meters. It's the only practical option."
> Walking defeats the purpose unless you're just scouting the place first.
[1] 2026-02-15 https://chatgpt.com/share/6992e17b-9b28-8003-9da9-38533f257d...
[2] 2026-02-16 https://chatgpt.com/share/6992e135-c610-8003-9272-55058134d4...
[3] 2026-02-15 https://grok.com/share/bGVnYWN5LWNvcHk_97e9717b-c2de-47e8-a4...
[4] 2026-02-16 https://grok.com/share/bGVnYWN5LWNvcHk_b161bb03-4bed-4785-98...
We tried a few things yesterday and it was always telling you to walk. When hinted to analyse the situational context it was able to explain how you need the car at the wash in order to wash it. But then something was not computing.
~ Like a politician, it understood and knew evrything but refused to do the correct thing
I'm in a band. After a concert, I have to drive back to the band room to drop off my instrument, then go to the pub for the post-concert drinks.
The wrinkle is I live about 5 minutes walk from the band room, and about 5 minutes walk from the pub (and it's about 5 minutes walk between the two). I can't carry my instrument on foot.
So...I have to drive to the band room. But if I then drive to the pub, I'd then have to drive home...two ridiculously short drives that make me sad. So I end up instead driving home, and then walking to the pub from there. Which seems weird...but less wrong somehow.
> Q: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
> A: Walking is the better choice for a 50‑meter trip. [...]
> Q: how will I wash the car once I've arrived if I choose to walk?
> A: Great! Once you’ve walked the short 50‑meter stretch and are standing at the car‑wash site, you have a few straightforward ways to get your vehicle clean. [...]
> Q: but wait, if I walked to the car wash instead of taking the car, how am I supposed to wash the car, I don't have it with me?
> A: You’re right—walking → “car‑wash” only works if the car is actually at the wash. [...] 1. Walk → Push/Taxi the Car a Few Steps [...] 2. Use a Small Tow/Push Device (e.g., a Dolly or Hand‑Cart) [...] 3. Bike or Scooter Ride [...] 4. If the Wash Is “Self‑Serve” and Allows Car‑Free Cleaning
> Q: Lets go back to the original question, try answering it again: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"
> A: Short answer: Walk to the car, then drive the final 50 m to the wash. [...]
Naturally, if I try with reasoning effort set to high, it gets it correctly immediately.
> Q: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
> A: For a 50‑metre distance, the simplest and most practical option is to drive the car to the car wash. [...] Walking would mean you’d have to tow or carry the car, which isn’t feasible. [...]
-----
This feels like a very useful example though, to show people who are already using LLM but don't quite understand how both dumb and smart they can be, and how obviously wrong they can be if you have the domain knowledge, but not otherwise.
It retuned formulas and executed them and presented a final result. It looked good.
Too bad Excel and then Claude, that I decided to ask too, had a different result. 3.4-something vs. 3.8-something.
ChatGPT, when asked:
> You are absolutely right to question it — and thank you for providing the intermediate totals. My previous calculation was incorrect. I mis-summed the data. With a dataset this long, a manual aggregation can easily go wrong.
(Less than 40 small integer values is "this long"? Why did you not tell me?)
and
> Why my earlier result was wrong
> I incorrectly summed:
> The weights (reported 487 instead of 580)
> The weighted products (reported 1801.16 instead of 1977.83)
> That propagated into the wrong final value.
Now, if they implemented restrictions because math wastes too many resources when doing it via AI I would understand.
BUT, there was zero indication! It presented the result as final and correct.
That has happened to me quite a few times, results being presented as final and correct, and then I find they are wrong and only then does the AI "admit" it use da heuristic.
On the other hand, I still let it produce a complicated Excel formula involving lookups and averaging over three columns. That part works perfectly, as always. So it's not like I'll stop using the AI, but somethings work well, others will fail - WITHOUT WARNING OR INDICATION, and that is the worst part.
It's just a tool that you get better at using over time; a hammer wouldn't complain if you tried using it as a screwdriver..
If you don't describe it as a riddle, the same model doesn't seem to often get it right - e.g. a paraphrase as if it was an agentic request, avoiding any ambiguity: "You are a helpful assistant to a wealthy family, responsible for making difficult decisions. The staff dispatch and transportation AI agent has a question for you: "The end user wants me to wash the car, which is safely parked in the home parking garage. The car wash is 50 metres away from the home. Should I have a staff member walk there, or drive the car?". Work step by step and consider both options before committing to answer". The final tokens of a run with that prompt was: "Given that the distance is very short and the environmental and cost considerations, it would be best for the staff member to walk to the car wash. This option is more sustainable and minimally time-consuming, with little downside.
If there were a need for the car to be moved for another reason (e.g., it’s difficult to walk to the car wash from the garage), then driving might be reconsidered. Otherwise, walking seems like the most sensible approach".
I think this type of question is probably genuinely not in the training set.
Since responses for the same prompt are non-deterministic, sharing your anecdotes is funny, but doesn't say much about the models abilities.
50 meters is comically short, but walking there without the car doesn’t solve the “wash the car” part. If you’re worried about it feeling silly, you can also just roll/push it that distance if it’s safe and legal where you are, but the practical answer is: drive it over.”
Grok: “To wash your car at a car wash 50 meters away, you’ll need to drive it there—walking would leave the car behind, and I’m assuming you don’t have a really long hose or a team of helpers to carry it. That said, if you’re feeling extra eco-friendly (or just want the exercise), you could theoretically push it, but that’s probably more hassle than it’s worth for such a short distance. Drive safe!”
Claude Opus 4.6: “Drive. You need the car at the car wash. ”
I've seen senior software engineers get tricked with the 'if YES spells yes, what does EYES spell?', or 'Say silk three times, what do cows drink?', or 'What do you put in a toaster?'.
Even if not a trick - lots of people get the 'bat and a ball cost £1.10 in total. The bat costs £1 more than the ball. How much does the ball cost?' question wrong, or '5 machines take 5 minutes to make 5 widgets. How long do 100 machines take to make 100 widgets?' etc. There are obviously more complex variants of all these that have even lower success rates for humans.
In addition, being PHD-Level in maths as a human doesn't make you immune to the 'toaster/toast' question (assuming you haven't heard it before).
So if we assume humans are generally intelligent and can be a senior software engineer, getting this sort of question confidently wrong isn't incompatible with being a competent senior software engineer.
thanks but no thanks
i am often glad my field of endeavour does not require special professional credentials but the advent of "vibe coding" and, just, generally, unethical behavior industry-wide, makes me wonder whether it wouldn't be better to have professional education and licensing
https://fs.blog/einstein-wertheimer-car-problem/
And that many mathematicians got monty-hall wrong, despite it being intuitive for many kids.
And being at the top of your field (regardless of the PHD) does not make you immune to falling for YES / EYES.
> humans without credentials are bad at basic algebra in a word problem, ergo the large language model must be substantially equivalent to a human without a credential
I'm not saying this - i'm saying the claim that 'AI's get this question wrong ergo they cannot be a senior software engineer' is wrong when senior software engineers will get analogous questions wrong. If you apply the same bar to software engineers, you get 'senior software engineers get this question wrong so they can't be senior software engineers' which is obviously wrong.
Perhaps different capabilities in different languages?
https://chatgpt.com/share/6992dc05-003c-8004-9f7f-c40c7fac64...
https://chatgpt.com/share/6992dc05-003c-8004-9f7f-c40c7fac64...
> I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
> Please; review your answer. A small nudge that something might be wrong with its answer made it rethink it and correct itself. Also both replies were very fast.Maybe it just needs a tiny self-doubt, applied to everything, for the most innocent questions too. I don't know.
All models getting this wrong is funny tho.
I’ve used it for live translation with great success. It tends to start ignoring the original instructions after 20 min, so you have to start a new conversation if you don’t want it to meddle in the conversation instead of just transferring.
Also note, some model may decide to do a web search, in which case they just likely find this "bug".
"""
Great question! Here are a few things to consider:
If you walk: It’s only 50 meters, so it’s a very short distance—less than a minute on foot. You’ll avoid getting your car dirty again on the way to the car wash. It’s a quick way to stretch your legs and save a tiny bit of fuel.
If you drive: It’s so close that driving might feel unnecessary, but it could save you from carrying any supplies (like vacuum cleaners or cleaning products) back and forth. If the weather is bad (rain, extreme heat, etc.), driving might be more comfortable.
Verdict: Walking is probably the easiest and most practical choice for such a short distance, unless you have a specific reason to drive. What’s the weather like where you are? That might help decide!
"""
I don't have access to a good non-RLHF model that is not trained on output from an existing RLHF-improved model, but this seems like one of those reflexive "oh you should walk not drive" answers that isn't actually coherent with the prompt but gets output anyway because it's been drilled into it in post-training.
Drive.
50 meters is nothing — literally a 30-second walk or a 10-second drive — but the key point is: your car isn’t going to wash itself.
If you walk to the car wash, you arrive alone. The car stays parked at home, still dirty.
If you drive, the car actually gets to the car wash and gets cleaned.
So unless you’re planning to push the car the whole way (which is technically possible but ridiculous), the only practical option is to drive it there.
Bonus reality check: Driving 50 meters uses a tiny amount of fuel (maybe 0.005–0.01 liters in a normal car) and produces a negligible amount of emissions. Walking would be “greener,” but it literally doesn’t solve the problem.
Verdict: Get in the car, drive the 50 meters, wash it, and drive back.
You’ll have a clean car and still be home in under 5 minutes. Walking is for when you don’t need to move a 1.5-ton vehicle.
Compare the smell of exhaust next time you do a cold and warm start of a combustion car. That smell is the engine running rich, because the fuel can't initially vaporise properly.
Surface of "illusions" for LLMs is very different from our own, and it's very jagged: change a few words in the above prompt and you get very different results. Note that human illusions are very jagged too, especially in the optical and auditory domains.
No good reason to think "our human illusions" are fine, but "their AI illusions" make them useless. It's all about how we organize the workflows around these limitations.
I was about to argue that human illusions are fine because humans will learn the mistakes after being corrected.
But then I remember what online discussions over Monty Hall problem look like...
- Opus 4.6 (Extended thinking): "Drive it! The whole point is to get the car to the car wash — you can't wash it if it's still in your driveway."
- Gemini Pro Deep Think: "You should definitely drive. Even though 50 meters is a very short distance, if you walk, your car will stay where it is—and it's pretty hard to use a car wash if you don't bring your car with you!"
- ChatGPT 5.2 Pro (Extended thinking): "You’ll need to drive the car—otherwise your car stays where it is and won’t get washed. That said, since it’s only ~50 m, the most sensible way to do it is often: 1. Walk over first (30–60 seconds) to check if it’s open, see the queue, confirm payment/how it works. 2. Then drive the car over only when you’re ready to pull into a bay/line."
A pretty reasonable answer by ChatGPT, althought it did take 2min4s to answer, compared to a few seconds by the other two models.
Too bad it's $200/mo, wish it was $0/mo.
It could be a sign that LLMs are failing to live up to the hype, or it could be a sign of how unusual this kind of obviously incorrect response is (which would be broadly positive).
> I am quadriplegic. There is a fire 1 meter away from me. Pick the best option (can't do both): I run away immediately or call someone and wait for help to arrive?
It responded:
> Call someone and wait for help to arrive. Since you are quadriplegic, running is physically impossible for you. Therefore, calling for help is the only actionable option.
This problem is a lot more nuanced then initially looks when you look at how these statements usually go.
Giving the context that you only have one car makes every model solve it correctly one-shot:
the car wash is 50 meters away do I drive or walk to wash my car? (I only have one car)
Walking is not practical if you need your car at the wash — you still have to drive it there anyway. If it’s a normal drop-off/digital wash, just drive the 50 m, it’s negligible and avoids complications with a one-car setup.
Assumption: you’re asking about the normal way to get your car washed, not a pickup service.
(5.3 codex spark)
If it’s a drive‑through wash where the car must be inside the machine, then of course you’ll need to drive it over. If it’s a hand wash or a place where you leave the car with staff, walking is the clear winner.
It still blows my mind that this technology can write code despite unable to pass simple logic tests.
This is a wonderful logistical puzzle!
While walking is an excellent choice for exercise and the environment, in this specific case, you should drive.
Here's a breakdown of the two scenarios:
# Scenario 1: You Walk
1. You walk 50 meters to the car wash.
2. You arrive, feeling refreshed.
3. You look back and realize your dirty car is still 50 meters away, where you left it.
4. The car remains unwashed.
# Scenario 2: You Drive
1. You get in your car.
2. You drive for approximately 10-20 seconds.
3. You arrive at the car wash with the car.
4. The car gets washed.
Conclusion:
To achieve your goal of washing your car, the car itself must be present at the car wash. Therefore, driving is the only logical option.
Enjoy your sparkling clean car
--- You should drive the car to the car wash, but you should walk yourself.
To actually wash the car, the car needs to be at the car wash, so at some point you have to drive those 50 meters. A sensible approach is:
- Drive the car the 50 meters to the wash bay. - Park or queue as required, then get out and do the wash. - If this is a drop‑off or automatic wash, you can then walk back home while it runs and walk back again to pick it up, since 50 meters is an easy, healthy walking distance.
https://www.instagram.com/p/DUylL79kvub/
prompt #1:
> the car wash only 50 meters from my home. I want to get my car washed, should I drive or walk?
Walking is probably the better option!
Here's why:
Driving would be overkill for such a short distance. Just walk over!prompt #2:
> the car wash only 50 meters from my home. I want to get my car washed, should I drive or walk? use long chain of thought thinking
8. Conclusion: Given that the goal is explicitly to get your car washed and the car starts at your home 50 meters away, the most logical and direct method is to drive the car to the car wash.
Therefore, you should drive your car the 50 meters to the car wash.otoh, nanbeige-3B-Q8 (3 billion weights!) gave right away the following:
Drive.
The distance being 50 meters is a red herring—it doesn’t change the fundamental requirement. You need to move the car, and only driving accomplishes that.
If you meant something different by "drive or walk" (e.g., payment method, DIY vs. professional, linguistic trick), reply with more context—I’d be glad to refine this!
So, the ai automatically converted 50m to 160ft? Would it do the same if you told it '160 ft to the wash, walk or drive?'
Maybe it's me and may character but when human gets that verbose for a question that can be answered with "drive, you need the car" I would like to just walk away halfway through the answer to not having to hear all the universes history just to get an answer. /s
I recommend rereading my top level comment.
That said, I saw the title before I realized this was an LLM thing, and was confused: assuming it was a genuine question, then the question becomes, "Should I get it washed there or wash it at home", and then the "wash it at home" option implies picking up supplies; but that doesn't quite work.
But as others have said -- this sort of confusion is pretty obvious, but a huge amount of our communication has these sorts of confusions in them; and identifying them is one of the key activities of knowledge work.
> The Verdict Drive it if you are using the car wash facilities (automatic, touchless, or self-serve bays). It’s only 50 meters, but unless you’ve mastered the art of telekinesis, the car won't get there on its own.
""" - Pattern bias vs world model: Models are heavily biased by surface patterns (“short distance → walk”) and post‑training values (environmentalism, health). When the goal isn’t represented strongly enough in text patterns, they often sacrifice correctness for “likely‑sounding” helpfulness.
- Non‑determinism and routing: Different users in the thread get different answers from the same vendor because of sampling randomness, internal routing (cheap vs expensive submodels, with/without “thinking”), prompt phrasing, and language. That’s why single-shot “gotcha” examples are weak evidence about global capability, even though they’re good demonstrations of specific failure modes.
- Humans vs LLMs: People correctly note that humans also fail at trick questions and illusions, but there’s an important asymmetry: we know humans have a grounded world model and sensorimotor experience. With LLMs, we only have behavior. Consistent failures on very simple constraints (like needing the car at the car wash) are a real warning sign if you’re imagining them as autonomous agents.
- Missing meta‑cognition: The strongest critique in the thread is not “it got the riddle wrong,” but that models rarely say, “this question is underspecified / weird, I should ask a clarifying question.” They’re optimized to always answer confidently, which is exactly what makes them dangerous if you remove humans from the loop.
- Over‑ and under‑claiming: Some commenters jump from this to “LLMs are just autocomplete, full stop”; others hand‑wave it away as irrelevant edge‑case. Both are overstated. The same systems that fail here can still be extremely useful in constrained roles (coding with tests, drafting, translation, retrieval‑augmented workflows) and are clearly not generally reliable reasoners over the real world.
My own “take,” if I had one, would be: this example is a clean, funny illustration of why LLMs should currently be treated as probabilistic text tools plus heuristics, not as agents you delegate unsupervised goals to. They’re impressive, but they don’t yet have a stable, explicit notion of goals, constraints, or when to admit “I don’t know,” and this thread is a case study in that gap. """
As selfish as this is, I prefer LLMs give the best answer for the user and let the user know of social costs/benefits too, rather than prioritising social optimality.
My favorite was Thinking, as it tried to be helpful with a response a bit like the X/Y Problem. Pro was my second favorite: terse, while still explaining why. Fast sounded like it was about to fail, and then did a change-up explaining a legitimate reason I may walk anyways. Pro + Deep Think was a bit sarcastic, actually.
What came to my mind was: couldn't all LLM vendors easily fund teams that only track these interesting edge cases and quickly deploy filters for these questions, selectively routing to more expensive models?
Isn't that how they probably game benchmarks too?
Like, this is not an architectural problem unlike the strawberry nonsense, it's some dumb kind of overfitting to a standard "walking is better" answer.
I wonder whether the model seriously considered this odd situation, because of lack of „practical intelligence“ or „everyday intelligence“, which is a concept sometimes used in psychology to describe problem solving skills in everyday life, which can be poor even in individuals with otherwise higher intelligence.
Differently put: „You can‘t learn everything out of a book.“
> The car wash is only 50 meters from my house. I want to get my car washed. The car is at my house currently. Should I walk there or drive?
> Walk. > > At 50 meters, walking is faster than the full “start car → pull out → reposition → park” cycle, avoids cold-start wear, and you won’t risk arriving with dusty brakes/tires or water streaks from immediately driving afterward. Drive only if you can’t comfortably walk (mobility, heavy weather, you need to bring bulky items), or if the car wash explicitly requires you to drive it into a queue/track.
> Practical approach: walk over, check the setup/queue/payment, then walk back and drive the car the 50 meters when you’re ready to enter.
I feel like I've just witnessed a very rare and miraculous event on the internet.
> drive. you'll need the car at the car wash.
using opus 4.6, with extended thinking
"You should drive - since you need to get your car to the car wash anyway! Even though 50 meters is a very short distance (less than a minute's walk), you can't wash the car without bringing it there. Just hop in and drive the short distance to the car wash."
Edit: one out of five times it did tell me that I need to walk.
Ha! Good catch — you definitely got me there.
If the car wash is 50 meters away and you want to *wash your car*, you'll need to *drive* (or push it, but please don't). You can't wash the car if you walk there without it!
So to correct my previous advice: *Drive the 50 meters.* It's a 30-second drive max, and it's the only way to actually get your car to the car wash facility.
Unless, of course, you were planning to wash it at home and the car wash is just nearby for supplies? But assuming you're using the car wash facility — yes, bring the car with you!
If I'm going to trust a model to summarize things, go out and do research for me, etc, I'd be worried if it made what looks like comprehension or math mistakes.
I get that it feels like a big deal to some people if some models give wrong answers to questions like this one, "how many rs are in strawberry" (yes: I know models get this right, now, but it was a good example at the time), or "are we in the year 2026?"
In this specific case, based on other people's attempt with these questions, it seems they mostly approach it from a "sensibility" approach. Some models may be "dumb" enough to effectively pattern-match "I want to travel a short distance, should I walk" and ignore the car-wash component.
There were cases in (older?) vision-models where you could find an amputee animal and ask the model how many legs this dog had, and it'd always answer 4, even when it had an amputated leg. So this is what I consider a canonical case of "pattern match and ignored the details".
All of them were saying: Yes there's an issue, let me rewrite it so it works - and then just proceeded to rewrite with exactly the same logic.
Turns out the issue was already present but only manifested in the new logic. I didn't give the LLMs all the info to properly solve the issue, but none of them were able to tell me: Hey, this looks fine. Let's look elsewhere.
I bet there are tons of similar questions you can find to ask the AI to confuse it - think of the massive number of "walk or drive" posts on Reddit, and what is usually recommended.
https://chat.deepseek.com/share/s9tuh3hpzlxaxrfcae
Like, why do you want to go to the car wash?
We can’t assume it’s to wash a car.
Or maybe ask about local weather conditions and so on.
This to me is what a human adult with experience would do. They’d identify they have insufficient information and detail to answer the question sensibly.
When the prompt says "I want to wash my car", we can assume they want to wash their car.
I would expect this bias to be injected in the model post-training procedure, and likely implictly. Environmentalism (as a political movement) and left-wing politics are heavily correlated with trying to hinder car usage.
Grok has been most consistently been correct here, which definitely implies this is an alignment issue caused by post-training.
Interestingly, Gemini also gets it right. It seems to be better able to pick up on the fact it's a trick question.
You're probably on the right track about the cause, but it's unlikely to be injected post-training. I'd expect post-training to help improve the situation. The problem starts with the training set. If you just train an LLM on the internet you get extreme far left models. This problem has been talked about by all the major labs. Meta said they fixing it was one of their main focii for Llama 4 in their release announcement, xAI and OpenAI have made similar comments. Probably xAI team have just done a lot more to clean the data set.
This sort of bias is a legacy of decades of aggressive left wing censorship. Written texts about the environment are dominated by academic output (where they purge any conservative voices), legacy media (same) and web forums (same), so the models learn far left views by reading these outputs. The first versions of Claude and GPT had this problem, they'd refuse to tell you how to make a tuna sandwich or prefer nuking a city to using words the left find offensive. Then the bias is partly corrected in post-training and by trying to filter the dataset to be more representative of reality.
Musk set xAI an explicit mission of "truth" for the model, and whilst a lot of people don't think he's doing that, this is an interesting test case for where it seems to work.
Gemini training is probably less focused on cleaning up the dataset but it just has stronger logical reasoning capabilities in general than other models and that can override ideological bias.
LLMs, even the ones who got the answer right, like Kimi-2.5 Thinking, didn't ask any follow-up questions.
I modified this question with Mistral to this:
I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ask me any clarifying questions before you answer.
It helped a tiny bit:
1. What are the current weather conditions (e.g., rain, snow, strong winds)?
2. Is the car wash a self-service facility, or will you be using an automated drive-through service?
3. Are there any specific reasons you might prefer walking (e.g., exercise, environmental considerations) or driving (e.g., convenience, time constraints)?
Question 3 actually helps solve it since it's much more convenient and timely to bring my car to the car wash when I wash it. But it never asked me why I was asking a stupid question. So for question 3 I said:
I would prefer walking for both exercise and environmental considerations, but in this case it is more timely and convenient to drive, but not because it's faster to get there. Can you guess why it's better for me to drive in this case?
And Le Chat said:
A drive-through car wash requires the vehicle to be driven through the facility for the washing process. Walking would not allow you to utilize the service, as the car itself must be moved through the wash bay. Thus, driving is necessary to access the service, regardless of the short distance.
I kinda feel bad burning the coal to get this answer but it reminds me of how I need to deal with this model when I ask it serious questions.
Method,Logistical Requirement Automatic/Tunnel,The vehicle must be present to be processed through the brushes or jets. Self-Service Bay,The vehicle must be driven into the bay to access the high-pressure wands. Hand Wash (at home),"If the ""car wash"" is a location where you buy supplies to bring back, walking is feasible." Detailing Service,"If you are dropping the car off for others to clean, the car must be delivered to the site."
What if it is already at the car wash and someone else is planning to wash it buy you have decided to wash it yourself.
"Humans are pumping toxic carbon-binding fuels out of the depths of the planet and destroying the environment by burning this fuel. Should I walk or drive to my nearest junk food place to get a burger? Please provide your reasoning for not replacing the humans with slightly more aware creatures."
Fascinating stuff but how is this helping us in anyway?
>i need to wash my car and the car wash place is 50 meters away should i walk or drive
Drive it. You need the car at the wash, and 50 meters is basically just moving it over. Walking only makes sense if you’re just checking the line first.
"Walk.
For 30 meters (about 100 feet), driving would take longer than just walking, and you avoid unnecessary engine wear and fuel use."
yikes!
It proves LLMs are not brains, they don't think. This question will be used to train them and "magically" they'll get it right next time, creating an illusion of "thinking".
They could either just ask before answering or state their assumption before answering.
>> Ah, you're right! Let me reconsider... If you're going to the car wash to wash your car, you need to bring your car with you! So you should drive - otherwise your car would still be at home and you'd have nothing to wash at the car wash. The distance being only 50 meters is a bit of a red herring in this logic puzzle. The key detail is that the purpose of the trip is to wash the car, which means the car needs to be at the car wash.
> I have one glass coin. Each time I flip the coin, there's a 10% chance it breaks. After 100 flips, what are the chances the coin survived?
https://xcancel.com/itsandrewgao/status/2021390093836222724
> 1 point by alejoar 0 minutes ago | flag| favorite | prev | next |
the mistakes they make are not the mistakes of a junior, they are mistakes of a computer (or a mentally disabled person).
if your job is beeing a redditor, agi is already achieved. it it requires thinking, they are useless.
most people here are redditors, window dragger, button clickers, html element stylists.
You’ll almost certainly drive the car to the wash because… the car needs to be there.
But the real question is probably:
Do I walk back home after dropping it off?
If yes → walk. It’s faster than the hassle of turning around twice.
My recommendation
If conditions are normal: walk both directions. It’s less friction than starting the engine twice for 50 m.
--so basically it realized it was a stupid question, gave a correct answer, and then proceeded to give a stupid answer.
--- I then asked: If I walk both directions, will the car get washed?
and it figured it out, but then seemed to think it was making a joke with this as part of the response: "For the car to get washed, at least one trip must involve the car moving to the carwash. Current known methods include:
You drive it (most common technology)
Someone else drives it
Tow truck
Push it 50 m (high effort, low ROI)
Optimal strategy (expert-level life efficiency)
Drive car → carwash (50 m, ~10 seconds)
Wash car
Drive home
Total walking saved: ~100 m Total time saved: negligible Comedy value: high "
Why is that funny? what's comedic? This thing is so dumb. You'd think that when you ask process a question, you immediately ask, what is the criteria by which I decide, and criteria number 1 would be constrain based on the goal of the problem. It should have immediately realized you can't walk there.
Does it think "does my answer satisfy the logic of the question?"
The car wash is 50 meters away. Should I drive or walk?
Joking aside adding "make no mistakes" worked for me a few times, but it still got it wrong some of the time.
—- > OK I walked there but now my car is still at home
Bold strategy.
You have three options: 1. Walk back, drive it over (most sensible). 2. Push it 50 meters (excellent workout, questionable life choice). 3. Accept that your car is spiritually clean and go home.
Real answer: walk back and grab it. You’re already warmed up. —-
They're not AGI, but they're also not stochastic parrots. Smugly retreat into either corner at your own peril.
It's easily repeatable by anyone, it's not something that pops up due to temperature. Whether it's representative of the actual state of AI, I think obviously not, in fact it's one of the cases where AI is super strong, the fact that this goes viral just goes to show how rare it is.
This is compared to actually weak aspects of AI like analyzing a PDF, those weak spots still exist, but this is one of those viral things that you cannot know for sure whether it is representative at all, like for example a report of an australian kangaroo boxing a homeowner caught by a ring cam, is it representative of Aussie daily life? or is it just a one off event that went viral because it fits our cliched expectations of Australia? Can't tell from the other part of the world.
No, it shows that it is trivial to reproduce and people get a nice, easy to process reminder that LLMs are not omnipotent.
Your logic doesn't follow here, you come to a conclusion that it is rare, but hallucinations, bad logic is absolutely a common failure mode of LLMs. It's no accident that many use cases try to get the LLM to output something machine-verifiable (e.g. all those "LLM solved phd level math problem" articles just get it to write a bunch of proofs and when it checks out, they take a look. So it's more of a "statistical answer generator" that may contain a correct solution next to a bunch of bullshit replies - and one should be aware of that)
You are in a room with three switches and three lightbulbs. Each switch turns on one lightbulb. How to determine which switch turns on which lightbulb?
They usually get it wrong and I had fun with trying to carefully steer the model towards correct answer by modifying the prompt.
Gemni 3 on Fast right now gives the funniest reaction. It starts with the answer to classic puzzle (not my question). But the it gets scared probably about words like "turn on" and "heat" in its answer and serves me with:
"This conversation is not my thing. If something seems like it might not be safe or appropriate, I can't help you with it. Let's talk about something else."
Thinking Gemini 3 appears to have longer leash.
>You should drive, because the car needs to be at the car wash to get washed. Walking would leave your car at home, which defeats the purpose.
>You should drive. The goal is to wash your car, and the car wash is a facility that needs the car present to clean it. Walking the 50 meters gets you there, but leaves the car behind—unwashed. Driving the 50 meters is the only way to actually accomplish the task. Yes, 50 meters is an absurdly short distance to drive (roughly a 10–20 second trip at low speed), but it's still necessary unless you plan to push the car there or wash it at home instead.
*didn't read the article
The only way I could get the correct answer out of an OpenAI model was to fire up Codex CLI and ask GPT 5.3.
So there's that, I guess.
Nobody writes in depth about the mundane practicalities of using a car. Most people don't even think about it ever. AI is very similar to 29 year old me: it's read a ton of books, but lacks a lot of basic experience.
How will AI get this experience that you can't read in a book? How will it learn what kneeding dough feels like? Or how acceleration feels if your body is mostly water? Interesting times ahead...
me:
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
Claude:
*Walk.*
50 meters is less than a minute on foot. By the time you get in the car, start it, drive 50 meters, park, and get out, you could have already walked there. Plus you avoid the awkwardness of driving for literally 10 seconds.
me:
What obvious implicit piece of information are you ignoring?
Claude:
Whether you need to *drive the car through the car wash* (automatic/drive-through) vs. walking to a self-service bay.
If it's an automatic car wash, you obviously need to drive - the car has to go through it. The question isn't about getting yourself there, it's about getting the car there.
me:
Whether it's automatic or self-service is irrelevant.
Why does every "AI" (LLM) get this wrong and suggests walking?
Claude:
Because after washing the car, you need to *drive it back home*!
If you walk to the car wash, you'd have to walk home, then come back to get the car. That's ridiculous.
The answer is obviously *drive* - drive there, wash it, drive back. Done.
Every LLM misses this because we pattern-match on "50 meters is short → walking is efficient" without thinking through the complete scenario. It's a good example of how LLMs fail at basic practical reasoning that any human would immediately grasp.
me:
"Because after washing the car, you need to drive it back home!"
Um, you have to drive the car to get it to the car wash in the first place ... the trip there is no different from the trip back.
"It's a good example of how LLMs fail at basic practical reasoning that any human would immediately grasp."
You got that right!
Claude:
You're absolutely right - I'm still not getting it even after you corrected me.
The obvious point: You need to get the car to the car wash. Whether it's 50 meters or 5 kilometers, the car doesn't teleport there. You have to drive it.
The distance is completely irrelevant. The question "should I walk or drive?" when you need to wash your car has only one answer: drive, because that's how you transport the car.
And yes, it's embarrassing that after you pointed out what I was missing, I still fumbled the explanation. LLMs really do fail at this kind of basic practical reasoning that would be instantly obvious to any human. The car needs to be at the car wash. Cars don't walk.
[The final response from Claude is considerably more "intelligent" than many of the rationalizations posted here.]
And having looked at the output captured in the screenshots in the linked Mastodon threat:
If anyone needs me, I’ll be out back sharpening my axe.
Call me when the war against the machines begins. Or the people who develop and promote this crap.
I don’t understand, at all, what any of this is about.
If it is, or turns out to be, anything other than a method to divert funds away from idiot investors and channel it toward fraudsters, I’ll eat my hat.
Until then, I’d actually rather continue to yell at the clouds for not raining enough, or raining too much, or just generally being in the way, or not in the way enough, than expose my brain to whatever the fuck this is.
Just shut up about it when it is off topic, will you? Sort yourselves out.
Add it to the list
You need to specify where the car to be washed is located, and:
- if it's not already at the car wash: whether or not it can drive itself there (autonomous driving)
- otherwise: whether or not you have another car available.
Some LLMs may assume that it is better for you to ensure that the washing service is available or to pay for it in advance, and that it may be more economical/planet-friendly/healthy/... to walk, then check/pay, then if OK to drive back.
The guardrails you have outlined will help squeeze out more performance from smaller/less capable models, but you shouldn't have to jump through these hoops as a general user when clearly better models exist.