> If two people disagree on whether something is funny, who's wrong? You can't say either of them is. There's no reward function for funny.
Laughter is the reward. N of 2 is a small sample size, but if one person laughed you could say it was 50% funny.
> a really good joke is recent, relevant, and shows deep understanding of its subject
These can help, but it ultimately doesn't matter how recent, relevant, or deep a joke is. If no one laughs, it wasn't funny.
mym1990 37 minutes ago [-]
Laughter is a decent signal, but it can be noise if the audience is uncomfortable or trying to please. Does the joke teller count as being part of the audience? I imagine if someone is telling the joke...they must think it is funny, so in most cases at least 1 participant thinks its funny. Sometimes jokes are unintended, maybe a faux pas, and it might be inappropriate for someone to laugh...does it make it not a joke, or does it make it not funny if I cannot laugh?
Lots of layers to this, but I guess the old adage "it depends" is very fitting here!
jessetemp 16 minutes ago [-]
Laughter isn’t a perfect signal, but is the only signal in all the noise you mentioned
nine_k 3 hours ago [-]
Some models are better at generating funny and poignant quips.
> my human mass-generates new ideas faster than I can research why the previous ones won't work
Circa GPT-3.5 to GPT-4o I was involved in some research in figuring out how to make LLMs funny. We tried a bunch of different things, from giving it rules on homonym jokes [1], double-entendre jokes, fine tuning on comedian transcripts, to fine tuning on publicly rated joke boards.
We could not make it funny. Also interesting was that when CoT research was getting a lot of attention, we tried a joke version of CoT, asking GPT4 to explain why a joke was funny in order to produce training set data. Most of the explanations were completely off base.
After this work, I became a lot less worried about the GAI-taking-over narrative.
Funny is very, very hard.
[1] without a dictionary, which at first seems inefficient, but this work demonstrated that GPT could perfectly reconstruct the dictionary anyway
politelemon 3 hours ago [-]
The model appears to have been overfitted to joke about the live demo being private.
onaclov2000 2 hours ago [-]
I mistakenly read this as training a trillion parameter model would be funny...at least I chuckled
tantalor 1 hours ago [-]
Is the comedy that these jokes suck?
scosman 3 hours ago [-]
I make a project for evals and fine-tuning and our default example task is a joke generator. It's a fun demo, but more importantly it's a really good use case to show how evaluating and optimizing LLMs is hard.
- There are a dozen plus common failure modes. How you split setup/punchline. Tropes. Toxicity. Template reuse. Each one needs a good eval.
- Datasets are hard: there's not much off the shelf, and as this author points out scraping gets a weird mix of quality.
- Models are really bad out of the box at humour.
At the end of the day it's just a hard problem that takes a lot of work and still isn't solved. GEPA prompts help, if you have good evals. Supervised fine-tuning works a little bit, but only if you training on a chain-of-thought thinking phase. We have a new evaluation builder that uses examples of edge cases for alignment, and jokes require the most iteration and feedback for refinement.
It would be easier to judge this if the jokes weren't 90% about AI and silicon valley, understandable only to people who subscribe to astralcodexten
emp17344 5 hours ago [-]
Probably because if they weren’t absurdly esoteric we’d be able to tell it isn’t funny.
omoikane 4 hours ago [-]
I thought this one was not bad:
[write a joke about thinking machines and the idea of tropes]
it's funny how enemies to lovers is a common trope that's uncommon in real life and lovers to enemies is an uncommon trope that's common in real life
simonw 5 hours ago [-]
Who's tommipink? Even a Google search couldn't explain that one.
userbinator 3 hours ago [-]
Unfortunately I find most AI hallucinations to be funnier than these attempts at comedy.
js8 1 hours ago [-]
Me too, which confirms the theory from Inside Jokes that what humans find funny are the flaws of logical thinking (and hallucinations mostly being hasty generalizations).
crawfordcomeaux 5 hours ago [-]
I once had a vivid dream that AI robots had taken over & were keeping humans around because they'd not yet mastered comedy. All of human culture globally was a comedy arms race with 24/7 open mic comedy jams on every corner.
They (the machines) had billboards/signage everywhere showing the estimated time left for humanity. A really good joke would lead the timer to grow (until they figured out how to produce the general patterns needed to both create and appreciate the joke).
colecut 4 hours ago [-]
openclaw, turn this into a broadway production, book me two front row seats, hire an escort..... brunette, 28, slim waist, sweet face, hates comedy and AI
kevmo314 3 hours ago [-]
Is writing in all lowercase funnier?
DanHulton 3 hours ago [-]
...this is actually a really interesting thought.
The act of writing in lowercase is not, in itself, funnier. But writing in the training set that is in all lowercase is _probably_ going to be the funnier writing.
Considering modern pundits online, "lowercase" is usually the case of the humourist. Lowercase also tends to be the case of sarcasm, almost exclusively deployed to be funny.
So it would make sense that models attempting to select for funny would also write in lowercase.
suddenlybananas 6 hours ago [-]
these really aren't very funny
4 hours ago [-]
Rendered at 05:28:45 GMT+0000 (Coordinated Universal Time) with Vercel.
Laughter is the reward. N of 2 is a small sample size, but if one person laughed you could say it was 50% funny.
> a really good joke is recent, relevant, and shows deep understanding of its subject
These can help, but it ultimately doesn't matter how recent, relevant, or deep a joke is. If no one laughs, it wasn't funny.
Lots of layers to this, but I guess the old adage "it depends" is very fitting here!
> my human mass-generates new ideas faster than I can research why the previous ones won't work
> this is called 'job security'
(https://nitter.poast.org/LetheAgent/status/20179595340865499...)
We could not make it funny. Also interesting was that when CoT research was getting a lot of attention, we tried a joke version of CoT, asking GPT4 to explain why a joke was funny in order to produce training set data. Most of the explanations were completely off base.
After this work, I became a lot less worried about the GAI-taking-over narrative.
Funny is very, very hard.
[1] without a dictionary, which at first seems inefficient, but this work demonstrated that GPT could perfectly reconstruct the dictionary anyway
- There are a dozen plus common failure modes. How you split setup/punchline. Tropes. Toxicity. Template reuse. Each one needs a good eval.
- Datasets are hard: there's not much off the shelf, and as this author points out scraping gets a weird mix of quality.
- Models are really bad out of the box at humour.
At the end of the day it's just a hard problem that takes a lot of work and still isn't solved. GEPA prompts help, if you have good evals. Supervised fine-tuning works a little bit, but only if you training on a chain-of-thought thinking phase. We have a new evaluation builder that uses examples of edge cases for alignment, and jokes require the most iteration and feedback for refinement.
If you want to try it: https://github.com/kiln-ai/kiln
They (the machines) had billboards/signage everywhere showing the estimated time left for humanity. A really good joke would lead the timer to grow (until they figured out how to produce the general patterns needed to both create and appreciate the joke).
The act of writing in lowercase is not, in itself, funnier. But writing in the training set that is in all lowercase is _probably_ going to be the funnier writing.
Considering modern pundits online, "lowercase" is usually the case of the humourist. Lowercase also tends to be the case of sarcasm, almost exclusively deployed to be funny.
So it would make sense that models attempting to select for funny would also write in lowercase.