They make LLMs play a very abstract game that rewards them points from answering the same as the other, and punishes them from answering differently, and LLMs tend to converge to an answer. From that to "social conventions" there is a long, long stretch. The paper lacks a baseline - wouldn't much simpler (non-LLM) systems also exhibit the same property? Is it that surprising that systems that are clones of each other (because they didn't even try "mixed societies" of different LLMs) agree when you give them points for agreeing?
Maybe I'm missing something but in my view this is pure hype and no substance. And note that I'm far from an LLM skeptic and I wouldn't rule out at all that current LLMs could develop social conventions, but this simulation doesn't really show that convincingly.
Crazy - this is saying agents develop their own biases and culture through their engagement with one another even if the individual agents are unbiased.
As obviously silly as this is, could it actually be useful for getting the observed phenomena acknowledged by people who might just tune out is presented with the underlying math that makes it individually?
How much overlap is there between people who think llms are magic and people who don't approve of applying math to groups of people? And how many in the overlap have positions where their opinions have outsized effects?
The twilight of Moore's law and diminishing returns for CPUs are driving a shift to GPUs and other accelerators. GPUs seem to do well for streaming/throughput type workloads.
What's interesting that Nvidia has managed to ride each of these bubbles so far.
Oh I thought this was going to be about the cult like in group signaling of LLM advocates, but this is a thing imagining language patterns as a society instead of language patterns of a society being a bias that the models have.
This sounds dismissive, but it is interesting in two more obvious ways:
1. The interaction appears to mimic human interaction online ( trendsetter vs follower )
2. It shows something some of us have been suggesting for a while: pathways for massive online manipulation campaigns
They make LLMs play a very abstract game that rewards them points from answering the same as the other, and punishes them from answering differently, and LLMs tend to converge to an answer. From that to "social conventions" there is a long, long stretch. The paper lacks a baseline - wouldn't much simpler (non-LLM) systems also exhibit the same property? Is it that surprising that systems that are clones of each other (because they didn't even try "mixed societies" of different LLMs) agree when you give them points for agreeing?
Maybe I'm missing something but in my view this is pure hype and no substance. And note that I'm far from an LLM skeptic and I wouldn't rule out at all that current LLMs could develop social conventions, but this simulation doesn't really show that convincingly.
Crazy - this is saying agents develop their own biases and culture through their engagement with one another even if the individual agents are unbiased.
As obviously silly as this is, could it actually be useful for getting the observed phenomena acknowledged by people who might just tune out is presented with the underlying math that makes it individually?
How much overlap is there between people who think llms are magic and people who don't approve of applying math to groups of people? And how many in the overlap have positions where their opinions have outsized effects?
I wonder when we will see LLMs being used to test economic theories.
Already exists in the current literature
It’s funny how we seem to be on this treadmill of “tech that uses GPUs to crunch data” starting with the Bitcoin thing, moving to NFTs, now LLMs.
Wonder what’s next.
The twilight of Moore's law and diminishing returns for CPUs are driving a shift to GPUs and other accelerators. GPUs seem to do well for streaming/throughput type workloads.
What's interesting that Nvidia has managed to ride each of these bubbles so far.
Accelerating the calculations done in probabilistic programming languages.
Any evidence this can be done, research literature-wise?
just a few picks in no particular order
https://pmc.ncbi.nlm.nih.gov/articles/PMC2945379/
https://arxiv.org/abs/2202.11154
https://lips.cs.princeton.edu/pdfs/liu2024generative.pdf
https://arxiv.org/abs/2501.05440
Oh I thought this was going to be about the cult like in group signaling of LLM advocates, but this is a thing imagining language patterns as a society instead of language patterns of a society being a bias that the models have.
This sounds dismissive, but it is interesting in two more obvious ways:
1. The interaction appears to mimic human interaction online ( trendsetter vs follower ) 2. It shows something some of us have been suggesting for a while: pathways for massive online manipulation campaigns
Oh !!