Category: Science
Kevin Bryan on LLMs and GPTs for economic research
Here is the talk. I am waiting for someone to do some background “anthropological” research and field work, and create a fully simulated economy of say a village of five hundred people. (It is not difficult to have LLMs simulate human responses in economic games.) After that, the social sciences will never be the same again.
New blog on science and economic growth
By economist Jack Leach, here is the blog. Here is a post on the Greek origins of modern science.
Lessons from the COVID War
In preparation for a National Covid Commission a group of scholars directed by Philip Zelikow (director of the 9/11 Commission) began interviewing people and organizing task forces (I was an interviewee). The Covid Commission didn’t happen, a fact that illustrates part of the problem:
The policy agenda of both major American political parties appear mostly undisturbed by this pandemic. There is no momentum to fix the system….The Covid war revealed a collective national incompetence in governance….One common denominator stands out to us that spans the political spectrum. Leaders have drifted into treating this pandemic as if it were an unavoidable national catastrophe.
The results of this early investigation, however, are summarized in Lessons from the COVID WAR. Overall, a good book, not as pointed or data driven as I might have liked (see my talk for a more pointed overview), but I am in large agreement with the conclusions and it does contain some clarifying tidbits such as this one on the Obama playbook.
Innumerable speeches, books, and articles have stated that the Obama administration gave the incoming Trump administration a “playbook” on how to confront a pandemic and that this playbook was ignored. The Obama administration did indeed prepare and leave behind the “Playbook for Early Response to High-Consequence Emerging Infectious Disease Threats and Biological Incidents.”
But this playbook did not actually diagram any plays. There was no “how.” It did not explain what to do…when it came to the job of how to contain a pandemic that was headed for the United States in January 2020, the playbook was a blank page.
I also appreciated that Lessons has some some unheralded success stories from the state and local level. You may recall Tyler and I blogging repeatedly in 2020 about the advantages of pooled tests. Eventually pooled testing was approved but I haven’t seen data on how widely pooling was adopted or the effective increase in testing capacity that was produced. Lessons, however, offers an anecdote:
In San Antonio, a local charitable foundation paired with a blood bank to create a central Covid PCR testing lab (antigen tests were not yet readily available) that could combine samples (pooling) for efficiency and cost reduction, but also determine which individual in a pool was positive. Importantly, results were available within about twelve hours. That meant results were available before the start of school the new day.
The program helped San Antonio get kids back into the schools.
More generally, it’s striking that US schools were closed for far longer than French, German or Italian schools. See data at right on the number of weeks that “schools were closed, or party closed, to in-person instruction because of the pandemic (from Feb. 2020-March 2022)”. (South Korea, it should be noted, had some of the most advanced online education systems in the world.)
One general point made in Lessons that I wholeheartedly agree with this is that the school closures and many of the other controversial aspects of the pandemic response such as the lockdowns and mask mandates “were really symptoms of the deep problem. Without a more surgical toolkit, only blunt instruments were left.” With better testing, biomedical surveillance of the virus and honest communication we could have done better with much less intrusive and costly policies.
Addendum: See my previous reviews of Gottlieb’s Uncontrolled Spread, Michael Lewis’s The Premonition, Slavitt’s Preventable and Abutaleb and Paletta’s Nightmare Scenario.
Addendum 2: A typo in Lessons had France closing schools for 2 weeks instead of 12 weeks. Corrected.
My Conversation with the excellent Kevin Kelly
Here is the audio, video, and transcript. Here is part of the summary:
…Kevin and Tyler start this conversation on advice: what kinds of advice Kevin was afraid to give, his worst advice, how to get better at following advice, and whether people who ask for advice really want it in the first place. Then they move on to the best places to see traditional cultures in Asia, the one thing in Kevin’s travel kit he can’t be without, his favorite part of India, why he’s so excited about brain-computer interfaces, how AI will change religion, what the Amish can teach us about tech adoption, the most underrated documentary, his initial entry point into tech, why he’s impressed by the way Jeff Bezos handles power, the last thing he’s changed his mind about, how growing up in Westfield, New Jersey affected him, his next project called the Hundred Year Desirable Future, and more.
Here is one excerpt:
COWEN: Do you ever feel that if you don’t photograph a place, you haven’t really been there? Does it hold a different status? Like you haven’t organized the information; it’s just out on Pluto somewhere?
KELLY: Yes, I did. When I was younger, I had a religious conversion and I decided to ride my bicycle across the US. And part of that problem — part of the thing was that I was on my way to die, and I decided to leave my camera behind for this magnificent journey of a bicycle crossing the US. It was the most difficult thing I ever did, because I was just imagining all the magnificent pictures that I could take, that I wasn’t going to take. I took a sketchbook instead, and that appeased some of my desire to capture things visually.
But you’re absolutely right. It was a little bit of an addiction, where the framing of a photograph was how I saw the world. Still images: I was basically, in my head, clicking — I was clicking the shutter at the right moments when something would happen. That, I think, was not necessarily healthy — to be so dependent on that framing to enjoy the world.
I’ve learned to wean myself off from that necessity. Now I can travel with just a phone for the selfies that you might want to take.
COWEN: Maybe the earlier habit was better.
Recommended. And here is Kevin’s new book Excellent Advice: Wisdom I Wish I’d Known Earlier.
A Systematic Review of Human Challenge Trials, Designs, and Safety
One of the most bizarre aspects of the COVID era was the institutional unwillingness to perform human challenge trials, which likely would have sped up vaccines and other treatments and saved lives. We let people join the military, indeed we advertise to encourage people to join the military, but for some reason running a human challenge trial is considered ethically fraught.
A new review find that HCTs are quite safe–more evidence that we have too few of these trials.
Human challenge trials (HCTs) are a clinical research method in which volunteers are exposed to a pathogen to derive scientifically useful information about the pathogen and/or an intervention [1]. Such trials have been conducted with ethical oversight since the development of the modern institutional review system of clinical trials in the 1970s. More recently, there has been renewed discussion about the ethical and practical aspects of conducting HCTs, largely fueled by interest in conducting HCTs for severe acute respiratory syndrome coronavirus 2. Past reviews of HCTs focused on reporting methods [2] and safety for single pathogens [3–6], but these did not explicitly evaluate the safety of HCTs by assessing reported adverse events (AEs) and serious adverse events (SAEs) across a range of pathogens. Furthermore, many additional HCTs have been performed since the publication of these reviews. To better inform discussions about future uses of HCTs, including during pandemic response, this article presents a systematic review of challenge trials since 1980 and reports on their clinical outcomes, with particular focus on risk of AEs and risk mitigation strategies.
Hat tip: Alec Stapp.
In Defense of Merit
An excellent paper co-authored by many luminaries, including two Nobel prize winners:
Merit is a central pillar of liberal epistemology, humanism, and democracy. The scientific enterprise, built on merit, has proven effective in generating scientific and technological advances, reducing suffering, narrowing social gaps, and improving the quality of life globally. This perspective documents the ongoing attempts to undermine the core principles of liberal epistemology and to replace merit with non-scientific, politically motivated criteria. We explain the philosophical origins of this conflict, document the intrusion of ideology into our scientific institutions, discuss the perils of abandoning merit, and offer an alternative, human-centered approach to address existing social inequalities.
Great work! The only problem? See where the paper was published (after being rejected elsewhere).
Emergent Ventures Africa and Caribbean, third cohort
Dr. Keabetswe Ncube is a Geneticist from South Africa. Her EV grant is for her work in using statistical and genetic inferences to help rural farmers maximize yields.
Frida Andalu is a petroleum engineer by training from Tanzania and a Ph.D. candidate. Her EV grant is to assist in her research of developing plant-based volatile corrosion inhibitors to mitigate top-of-line corrosion in natural gas pipelines.
Desta Gebeyehu is a biochemical researcher from Kenya. Her EV grant is to assist in her research of developing bioethanol-gel fuel from organic waste.
Bobson Rugambwa is a software engineer from Rwanda. After graduating with a master’s from Carnegie Mellon University he co-founded MVend to tackle the problem of financial inclusion in Rwanda.
Sylvia Mutinda is a Chemist and Ph.D. researcher from Kenya. Her EV grant is to assist with her search on strigolactone biosynthesis focusing on countering striga parasites in sorghum farms in Kenya.
Dr. Lamin Sonko, born in the Gambia and raised in the U.S., is an Emergency Medicine physician and recent Wharton MBA graduate. He is the founder of Diaspora Health, an asynchronous telemedicine platform focused on patients in the Gambia and Senegal.
Cynthia Umuhire is an astronomer from Rwanda and Ph.D. researcher. She works as a space science analyst at the Rwanda Space Agency. Her EV grant is to assist her in establishing a knowledge hub for junior African researchers in space science.
Brian Kaaya is a social entrepreneur from Uganda. He is the founder of Rural Solars Uganda, a social enterprise enabling rural households in Uganda to access electricity through affordable solar panels.
Shem Best is a designer and urban planning enthusiast from Barbados. His EV grant is to start a blog and podcast on urban planning in the Caribbean to spur discourse on the built environment in the Caribbean and its impact on regional integration.
Susan Ling is an undergraduate researcher from Canada. Her EV grant is to continue her research on biodegradable, long-acting contraceptive implants with a focus on Africa, and general career development
Elizabeth Mutua is a computer scientist and Ph.D. researcher from Kenya. Her EV grant is to assist in her research on an efficient deep learning system with the capacity to diagnose retinopathy of prematurity disease.
Youhana Nassif is the founder and director of Animatex, the biggest animation festival in Cairo, Egypt. His EV grant is for the expansion of the festival and general career development.
Esther Matendo is a Ph.D. candidate in food science from the Democratic Republic of the Congo. Her EV grant is to assist in her research on plant-based treatments of mycotoxin contamination on maize in South Kivu (one of the main maize production zones in the DRC).
Alex Kyabarongo is a recent graduate of veterinary medicine from Uganda. He is now a political affairs intern at the Implementation Support Unit of the Biological Weapons Convention at the United Nations Office for Disarmament Affairs in Geneva. His EV grant is for general career development.
Margaret Murage is a Ph.D. researcher from Kenya. Her EV grant is to assist in her research of developing new photosensitizing agents for photodynamic therapy for cancer treatment.
Kwesiga Pather, for design and development of low-cost drones for agricultural uses in Uganda and general career development.
Dr. Sidy Ndao is a materials engineer by training from Senegal. He is the founder and President of the Dakar American University of Science and Technology (DAUST). The university provides a rigorous American-style English-based engineering education to African students.
Chiamaka Mangut is a Ph.D. candidate at Columbia University from Nigeria. Her EV grant is to fund new field research using archaeobotanical methods to study ancient populations in the Jos Plateau.
Dr. Yabebal Fantaye is Cosmologist by training from Ethiopia. He is the co-founder of 10 Academy, a training bootcamp to assist recent graduates of quant fields to acquire remote data science-related jobs.
For his very good work on these award I wish to heartily than Rasheed Griffith. And here is a link to the previous cohort of Africa winners.
The first recorded scientific grant system?
“Encouragements” from the French Académie des Sciences, 1831-1850.
The earliest recorded grant system was administered by the Paris-based Académie des Sciences following a large estate gift from Baron de Montyon. finding itself constrained in its ability to finance the research of promising but not-well-established savants, the academy seized on the flexiblity afford by the Montyon gift to transform traditional grands prix into “encouragements”: smaller amounts that could broaden the set of active researchers. Even though the process was highly informal (the names of the early recipients were not published in the academy’s Compte rendus), it apparently avoided suspected or actual cases of corruption…Throughout the 19th century, however, the academy struggled to convince wealthy donors to abandon their preference for indivisible, large monetary prizes in favor of these divisible encouragements.
That is from the Pierre Azoulay and Danielle Li essay “Scientific Grant Funding,” in the new and highly useful NBER volume Innovation and Public Policy, edited by Austan Goolsbee and Benjamin F. Jones. (But according to the book’s own theories, shouldn’t the book be cheaper than that?)
Executive in Residence, Math Talent Search
Who We Are: Carina Initiatives
Carina Initiatives is a philanthropic fund working to send more kids from more communities to the frontiers of science and technology. We see math as fundamental to future innovation; as such, we fund and support organizations that work to inspire, unearth, and train math talent.
Here is the link. This has the promise to be an important post, but it needs the right person.
A Mosquito Factory?!
A “mosquito factory” might sound like the last thing you’d ever want, but Brazil is constructing a facility capable of producing five billion mosquitoes annually. The twist? The factory will breed mosquitoes carrying a special bacteria that significantly reduces their ability to transmit viruses. As far as I can tell, however, the new mosquitoes still suck your blood.
Nature: The bacterium Wolbachia pipientis naturally infects about half of all insect species. Aedes aegypti mosquitoes, which transmit dengue, Zika, chikungunya and other viruses, don’t normally carry the bacterium, however. O’Neill and his colleagues developed the WMP mosquitoes after discovering that A. aegypti infected with Wolbachia are much less likely to spread disease. The bacterium outcompetes the viruses that the insect is carrying.
When the modified mosquitoes are released into areas infested with wild A. aegypti, they slowly spread the bacteria to the wild mosquito population.
Several studies have demonstrated the insects’ success. The most comprehensive one, a randomized, controlled trial in Yogyakarta, Indonesia, showed that the technology could reduce the incidence of dengue by 77%1, and was met with enthusiasm by epidemiologists.
In Brazil, where the modified mosquitoes have so far been tested in five cities, results have been more modest. In Niterói, the intervention was associated with a 69% decrease of dengue cases2. In Rio de Janeiro, the reduction was 38%3.
Wolbachia-infected mosquitoes have already been approved by Brazilian regulatory agencies. But the technology has not yet been officially endorsed by the World Health Organization (WHO), which could be an obstacle to its use in other countries. The WHO’s Vector Control Advisory Group has been evaluating the modified mosquitoes, and a discussion about the technology is on the agenda for the group’s next meeting later this month.
Do older economists write differently?
The scholarly impact of academic research matters for academic promotions, influence, relevance to public policy, and others. Focusing on writing style in top-level professional journals, we examine how it changes with age, and how stylistic differences and age affect impact. As top-level scholars age, their writing style increasingly differs from others’. The impact (measured by citations) of each contribution decreases, due to the direct effect of age and the much smaller indirect effects through style. Non-native English-speakers write in different styles from others, in ways that reduce the impact of their research. Nobel laureates’ scholarly writing evinces less certainty about the conclusions of their research than that of other highly productive scholars.
Here is the full NBER paper by Lea-Rachel and Daniel S. Hamermesh.
Strong and Weak Link Problems and the Value of Peer Review
Adam Mastroianni’s has an excellent post on strong-link vs weak-link problems in science. He writes:
Weak-link problems are problems where the overall quality depends on how good the worst stuff is. You fix weak-link problems by making the weakest links stronger, or by eliminating them entirely.
Food safety is a weak link problem, bank or computer security is a weak-link problem, many production processes are weak-link, also called O-ring problems.
[But] some problems are strong-link problems: overall quality depends on how good the best stuff is, and the bad stuff barely matters….Venture capital is a strong-link problem: it’s fine to invest in a bunch of startups that go bust as long as one of them goes to a billion.
….Here’s the crazy thing: most people treat science like it’s a weak-link problem.
Peer reviewing publications and grant proposals, for example, is a massive weak-link intervention. We spend ~15,000 collective years of effort every year trying to prevent bad research from being published. We force scientists to spend huge chunks of time filling out grant applications—most of which will be unsuccessful—because we want to make sure we aren’t wasting our money.
These policies, like all forms of gatekeeping, are potentially terrific solutions for weak-link problems because they can stamp out the worst research. But they’re terrible solutions for strong-link problems because they can stamp out the best research, too. Reviewers are less likely to greenlight papers and grants if they’re novel, risky, or interdisciplinary. When you’re trying to solve a strong-link problem, this is like swallowing a big lump of kryptonite.
At Maximum Progress, Max Tabarrok has some nice diagrams illustrating the issue:
If you have a weak-link view of science, you’d think peer review works something like this. The relationship between quality and eventual impact is linear, or perhaps even bowed out a bit. Moving resources from low input quality projects to average ones is at least as important to eventual impact as moving resources from average projects to high quality ones.
In a strong-link model of science, filtering the bottom half of the quality distribution is less important to final impact [because the impact of research is highly non-linear].
Even though peer review has the same perfect filter on the quality distribution, it doesn’t translate into large changes in the impact distribution. Lots of resources are still being given to projects with very low impact. Although the average input quality increases by the same amount as in the weak link model, the average final impact barely changes. Since peer review has significant costs, the slightly higher average impact might fail to make up for the losses in total output compared to no peer review.
This is a simplified model but many of the simplifying assumptions are favorable for peer review. For example, peer review here is modeled as a filter on the bottom end of the quality distribution…But if peer review also cuts out some projects on the top end, its increase of the average impact of scientific research would be muted or even reversed.
Eight Things to Know about LLMS
A good overview from computer scientist Samuel R. Bowman of NYU, currently at Anthropic:
1. LLMs predictably get more capable with increasing investment, even without targeted innovation.
2. Many important LLM behaviors emerge unpredictably as a byproduct of increasing investment.
3. LLMs often appear to learn and use representations of the outside world.
4. There are no reliable techniques for steering the behavior of LLMs.
5. Experts are not yet able to interpret the inner workings of LLMs.
6. Human performance on a task isn’t an upper bound on LLM performance.
7. LLMs need not express the values of their creators nor the values encoded in web text.
8. Brief interactions with LLMs are often misleading.
Bowman doesn’t put it this way but there are two ways of framing AI risk. The first perspective envisions an alien superintelligence that annihilates the world. The second perspective is that humans will use AIs before their capabilities, weaknesses and failure modes are well understood. Framed in the latter way, it seems inevitable that we are going to have problems. The crux of the dilemma is that AI capability is increasing faster than our AI understanding. Thus AIs will be widely used long before they are widely understood. You don’t have to believe in “foom” to worry that capability and control are rapidly diverging. More generally, AIs are a tail risk technology, and historically, we have not been good at managing tail risks.
What do we need to talk to whales?
We detail a scientific roadmap for advancing the understanding of communication of whales that can be built further upon as a template to decipher other forms of animal and non-human communication. Sperm whales, with their highly developed neuroanatomical features, cognitive abilities, social structures, and discrete click-based encoding make for an excellent model for advanced tools that can be applied to other animals in the future. We outline the key elements required for the collection and processing of massive datasets, detecting basic communication units and language-like higher-level structures, and validating models through interactive playback experiments. The technological capabilities developed by such an undertaking hold potential for cross-applications in broader communities investigating non-human communication and behavioral research.
That is from a new research paper by Jacob Andreas, et.al., and the (ungated) article offers considerable detail on exactly how to do this. They already have funding from both Dalio and Audacious.
From the comments, on AI safety
This is from Richard Ngo, who works on the governance team at OpenAI:
A few points:
1. I agree that the alignment community has generally been remiss in not trying hard enough to clarify the arguments in more formal papers.
2. The only peer-reviewed paper making the case for AI risk that I know of is: https://onlinelibrary.wiley.com/doi/10.1002/aaai.12064. Though note that my paper (the second you linked) is currently under review at a top ML conference.
3. I don’t think that a formal model would shed much light here. My goal in writing my paper was to establish misaligned power-seeking AGI as a credible scientific hypothesis; I think that most who think it’s credible would then agree that investigating it further should be a key priority, whether or not their credences are more like 10% or more like 90%.
From this batch of comments. Here is Richard on Twitter.