(highlight:: Explore v.s. Exploit: Finding Solutions Quickly Can Get You Stuck in a Local Optimum
Summary:
The explore, exploit trade-off in computer science highlights the challenge of simultaneously optimizing for effectively doing things (exploit) and search through all the possibilities (explore).
When trying to find new solutions, one can either search for solutions similar to the ones already known, making small changes to accommodate new evidence quickly, or consider the possibility of a much better solution that is further away in a high dimensional space. While the former allows for quickly finding a pretty good solution, the latter may lead to finding a much better solution, albeit in a space too large to search systematically.
Transcript:
Speaker 1
So when I started doing the work in AI, one of the really, very, very general ideas that comes across again and again in computer science is this idea of the explore, exploit trade on. And the idea is that you can't get a system that is simultaneously going to optimize for actually being able to do things effectively. That's the exploit part. And being able to figure out, search through all the possibilities. So let me try to describe it this way. I guess we're a podcast. So you're going to have to imagine this usually I wave my arms around a lot here. So imagine that you have some problem you want to solve or some hypothesis that you want to discover. And you can think about it as if there's a big box full of all the possible hypotheses and all the possible solutions to your problem or possible policies that you could have, for instance, Your reinforcement learning context. And now you're in a particular space in that box. That's what you know now. That's the hypotheses you have now. That's the policies you have now. Now what you want to do is get somewhere else. You want to be able to find a new idea, a new solution. And the question is how do you do that? And the idea is that there are actually two different kinds of strategies you could use. One of them is you could just search for solutions that are very similar to the ones you already have. And you could just make small changes in what you already think to accommodate new evidence or a new problem. And that has the advantage that you're going to be able to find a pretty good solution pretty quickly. But it has a disadvantage. And the disadvantage is that there might be a much better solution that's much further away in that high dimensional space. And any interesting space is going to be too large to just search completely systematically. You're always going to have to choose which kinds of possibilities you want to consider. So it could be that there's a really good solution, but it's much more different from where you currently are. And the trouble is that if you just do something like what's called hill climbing, you just look locally, you're likely to get stuck in what's called a local optimum.)
- Time 0:10:40
- child_development, explore_vs_exploit, learning, local_optimum, snipdpost-queue,

(highlight:: Children Optimize for Explore Instead of Exploit
Summary:
The process of exploring and narrowing down possibilities is likened to a temperature metaphor, where a high temperature search involves random and broad exploration, similar to the behavior of a four-year-old.
This broad search allows for exploring various possibilities and learning as much as possible, even though it may involve trying out strange and unhelpful things. Children are found to make high temperature explore decisions compared to adults, which shows their inclination towards exploration and learning.
Transcript:
Speaker 1
And the way that that typically gets resolved in various kinds of forms is start out with these big broad search through lots and lots of possibilities. Jump around from one possibility to another. And then slowly cool off the narrow down. And the metaphor that's often used is this metaphor about temperature. So you could think about big boxes if it had air molecules in it instead of hypotheses. A low temperature search would be just a search where you weren't moving very much. The high temperature search would be this big, much easier, more random, bouncy kind of search. And I like to say sometimes for anyone who writes a four-year-old at home, which of those sounds more like you're a four-year-old? Four-year-olds are both literally and metaphorically noisy and bouncy. So the solution is still it with this big broad search. The disadvantage of course is that you're not, you might be spending time trying out really weird strange things that aren't going to help you very much. And then when you see something that looks like it's in the right ballpark narrow in to the cooler solution. So it's like what happens in metallurgy with a kneeling where you heat up a metal first and then gradually cool it to end up with a more robust metal. Okay, sorry, that's a long description. But of course, if you're thinking about childhood from that perspective, from the perspective of that kind of explore exploit contrast or from the perspective of high temperature, Low temperature, and kneeling, then a lot of the things that look like, oh, this turn out to actually be features. So actually doing a lot of random variability, being noisy, having a broad focus of attention instead of a narrow focus of attention. All those things that are really not good from the exploit perspective when you want to do is just implement that policy, say as quickly and effectively as you can. Those things all turn out to be real benefits from the explore perspective of what you want is to learn as much as you can about the world and explore as many possibilities as you can. And what my lab and a bunch of other labs recently have been doing is showing that you can show even formally that children are making those kind of high temperature explore decisions Compared to apps.)
- Time 0:12:52
- child_development, explore_vs_exploit, snipddont-post,

(highlight:: Understanding Populations As Groups of Explorers and Exploiters
Summary:
Populations can be understood through the lens of explorers and exploiters, where some individuals or groups are dedicated to exploring new opportunities while others focus on exploiting existing resources.
This division of labor is observed in various species, such as honeybees, where scouts explore and are supported by other bees. Similarly, in human societies, scientists can be compared to the explorers, playing a crucial role in uncovering new knowledge and opportunities for the community.
This developmental strategy offers an advantage by minimizing free rider problems and promoting the survival of the organism.
It also alleviates some of the group selection issues faced by entire societies when considering the balance between exploration and exploitation.
Transcript:
Speaker 1
And science in a way is a nice example of this. So for example, you could have a, instead of having a developmental division of labor where my young self is exploring and I'm exploiting, you could have a division of labor division Of labor where some people are exploring and some people are exploiting. And that seems to be what insects are doing, for example. So for honeybees, you have different kinds of rolls of the scouts and the scouts are being fed by everybody else. Like they're being nurtured by the workers. Like you still have the same kind of trade-off there for the whole hive between the exploration and the exploitation. And I think to some extent in human societies when you end up with institutions like science, the scientists are kind of like the honeybee scouts. They're given a special role of functioning like the children actually explore. The developmental strategy has a nice advantage though, which is that you don't have to sort worry about free riders because you're the same organism, right? So more exploration is going to be used to help you from an evolutionary point of view. It's going to be used to help you continue to survive in the future. So you don't have to worry quite as much about some of the group selection problems that you have if you're trying to think about an entire society.)
- Time 0:27:52
- collaboration, collective_intelligence, explore_vs_exploit, sociology, snipddont-post,

(highlight:: Tempering Metal: Attempts to Increase Brain Plasticity in Adults Are Challenging
Summary:
Induction of plasticity by psychedelics and other practices can be therapeutic but needs to be followed by a cool-off period.
Religious practices, mysticism, and meditation are examples of activities that can put adults back into a state of plasticity similar to that of children. Integration with therapy and making sense of experiences are important for utilizing induced plasticity effectively.
The real world progresses by having new generations exploring and making sense of the environment and applying their discoveries to solve new problems.
Transcript:
Speaker 1
So you mentioned NMDA and I think there's pretty general consensus that the mechanism by which say psychedelics are therapeutic is through this induction of plasticity that that Seems to be essentially what chemicals do. And again, the problem is once you've induced plasticity, once you're in this high temperature state, how do you cool off and where do you cool? And I think independently of chemicals, things like religious practices, mysticism, meditation, those are all examples of things that people have done as long as people have been Around as adults that have the effect of putting them back into this kind of state of plasticity of children. But again, the problem is you can't be in that state indefinitely as an adult. You need to be able to also be in the cooler export state. And what happens, which is why things like integration with psychedelics therapy seems to be so important, you need to be able to make sense out of the experiences that you have in order To actually make them be useful or helpful to you as you're going on in life. But I do think that the real world has always been done and will continue to be done by having new generations of children, by having new generations of young people who are coming in, Seeing the new world and new environment from scratch with this kind of broad exploration and making sense out of it and then taking the things that they make sense of and they discover And applying them to the next set of problems that they're going to have to solve.)
- Time 0:36:52
- creativity, knowledge_acquisition, learning, neuroplasticity,

(highlight:: AI as a Cultural Technology: Enabling Unprecedented Information Flow Between Generations
Summary:
Chat GPT should be viewed as a cultural technology rather than an intelligent agent, enabling the transmission of information across generations.
Cultural revolution, facilitated by children, is considered a distinctive form of intelligence as it allows each generation to inherit and utilize information from previous generations without having to rediscover it. The evolution of language and writing has facilitated the more effective transmission of information across time and space.
These cultural technologies have had deep transformative effects on society, such as the printing technology in the 18th century that influenced the American and French Revolutions through the spread of democracy and enlightenment ideas, albeit with different outcomes.
Transcript:
Speaker 1
But what I've argued is instead of thinking about something like chat GPT as if it was an agent and then debating whether it's an intelligent agent or not, the right way to think about It is this kind of cultural technology, what do I mean by cultural technology? We think about things like writing, print, language, libraries, all these kinds of techniques and technologies we have that enable us to take information from many, many different People and give it to a new generation. So lots of people have argued, and I think this is probably right, that cultural revolution is one of our most distinctive forms of intelligence. Again, children are the ones who are doing this. The fact that each generation of children can take all the information that all those previous generations have discovered and use it themselves without having to rediscover it. That's like a great human superpower. And one of the interesting things that we've done really, again, since the evolution of language is to find new ways of making that transmission of information from one person to another More effective. So if you think about the difference between writing and speaking an oral language, when you have writing, then you can get information not just from the people who are leaving your Immediate preview, but from people way off in the past, from people in many different places, distant in space and in time. And what's happened is I think there's a very good case to be made that those cultural technologies just time and again have had really deep transformative effects on our society. An example I like is that there were these changes in printing technology in the 18th century that made it much easier for basically anybody could go out and get a printing press and print Pamphlets and distribute them. And those technology was really responsible for the American Revolution. A lot of the ideas about democracy and the enlightenment got spread through these pamphlets. On the other hand, as the great historian Robert Darden has pointed out, in France, that same technology led to this just absolute spew of libel and obscenity and things that make Twitter And Facebook even if their words look pretty tame by comparison also led to the distribution of ideas about democracy and enlightenment, but it ended up in France taking a much less Beneficent form than it did in America.)
- Time 0:44:01
- knowledge_transfer,

(highlight:: AI Can Help Uncover What Ideas and Approaches Are/Aren't Novel
Summary:
AI, such as Dolly, can help artists identify clichés to avoid in their work by recognizing commonly used images and traits, allowing artists to intentionally steer clear of unoriginal concepts.
This can serve as a valuable aid for artists seeking to create original and innovative work, although the challenge lies in avoiding the temptation to simply replicate what has already been done.
Transcript:
Speaker 1
I was talking about exactly this sort of problem with my brother who's an art critic. And he quoted an artist friend of his who said, oh, these things like Dolly are wonderful because they will immediately tell you here's the cliché that you should avoid. So if Dolly can generate it, that means it's looked out and it's found all the images that all those traits, illustrations are using. And here's like a summary of the most trait, boring cliché thing that you can imagine. So if you're really an artist, make sure you avoid that. And she was saying, of course, that's the hardest thing to do is to avoid just copying them and all stuff that everybody else has done. So she thought Dolly was a really great aid from that perspective. And I think that's generally true.)
- Time 0:46:34
- artificial_intelligence_ai, content_creation, creativity, ideas, innovation, novelty, writing,

(highlight:: Parenting & Tech: "The Day After Your Children Are Born Is Mad Max"
Summary:
The day before you're born is likened to Eden, while the day after your children are born is described as Mad Max, signifying significant technological changes and innovations within one's lifetime, especially after adulthood.
Transcript:
Speaker 1
And it was a scientific version of a point that I've sometimes phrased by saying the day before you're born is Eden and the day after your children are born is Mad Max. So everybody seems to think that the things that happened before they were born, that's not technology, that's just life, right? But of course, the things that happen within your lifetime, especially after you're an adult, those are big technological changes in innovations.)
- Time 0:49:11
- differential_technological_development, parenting, technology,

(highlight:: "We Need a Circuit Breaker for Emerging Tech": Parallels Between the Tech Regulation Across Time
Summary:
Imagine if electricity was introduced today and had the potential to burn down houses.
The insurance industry stepped in with circuit breakers to prevent accidents. Similarly, we need regulations or 'code' for platforms like Twitter and Facebook to ensure they are productive rather than harmful.
It will require dedicated effort to make these technologies beneficial.
Transcript:
Speaker 1
Another example that I like is think about electricity. So again, this is one of those ones where thinking about it from the perspective of, you know, so putting yourself in the past. There's someone thinking, said, we have this idea of this thing that we want to put in everybody's house except that we know that it gets hot enough to burn down houses. But we still think it would be really useful to put it in everybody's house. So let's go ahead and do that. Well, it turns out that the reason why we can have that powerful force at everybody's house and houses aren't burned down all the time is because the insurance industry said, oh no, we've Invented this thing called a circuit breaker and you can only get your house insured if you put electricity in if you make sure that there's a circuit breaker there. I have a son who's a carpenter and, you know, he'll show you the book of code that's this thick about what you have to do to build a house. So the problem is that we haven't got like the code or the circuit breaker is for something like Twitter or Facebook. And I think that we will. It's just going to be a matter of, but it's not like it will just happen without anybody trying. People are going to have to work very hard to figure out how to make those technologies be productive versus non-productive.)
- Time 0:55:21
- policy, regulation, technology,

(highlight:: Rethinking The Business Model of Cultural Technologies
Summary:
The central problem lies in the business model of catching attention through advertising, which leads to negative consequences.
Cultural technologies should be treated more like public libraries or public utility models, rather than being driven by a big advertising agency.
Transcript:
Speaker 1
I think the real central problem is that we have a business model that is people have pointed out in some amplifying because the business model is catching attention through advertising. There's no reason why the net has to be designed according to that business model and that business model ends up having a lot of negative consequences. And again, think about the internal question engine. In fact, there was a particular business model behind that. It certainly had a lot to do with its consequences for good or ill. I think an idea that a lot of people have had and just needs to be, you need to figure out how to implement it, is think about some of these things as being more like public libraries or other Kinds of more public utility kind of model for a lot of what we do with these cultural technologies rather than thinking about them as basically a big advertising agency.)
- Time 1:03:03
- business_models, cultural_technologies, incentives, technology,