Category: Science
A Systematic Review of Human Challenge Trials, Designs, and Safety
One of the most bizarre aspects of the COVID era was the institutional unwillingness to perform human challenge trials, which likely would have sped up vaccines and other treatments and saved lives. We let people join the military, indeed we advertise to encourage people to join the military, but for some reason running a human challenge trial is considered ethically fraught.
A new review find that HCTs are quite safe–more evidence that we have too few of these trials.
Human challenge trials (HCTs) are a clinical research method in which volunteers are exposed to a pathogen to derive scientifically useful information about the pathogen and/or an intervention [1]. Such trials have been conducted with ethical oversight since the development of the modern institutional review system of clinical trials in the 1970s. More recently, there has been renewed discussion about the ethical and practical aspects of conducting HCTs, largely fueled by interest in conducting HCTs for severe acute respiratory syndrome coronavirus 2. Past reviews of HCTs focused on reporting methods [2] and safety for single pathogens [3–6], but these did not explicitly evaluate the safety of HCTs by assessing reported adverse events (AEs) and serious adverse events (SAEs) across a range of pathogens. Furthermore, many additional HCTs have been performed since the publication of these reviews. To better inform discussions about future uses of HCTs, including during pandemic response, this article presents a systematic review of challenge trials since 1980 and reports on their clinical outcomes, with particular focus on risk of AEs and risk mitigation strategies.
Hat tip: Alec Stapp.
In Defense of Merit
An excellent paper co-authored by many luminaries, including two Nobel prize winners:
Merit is a central pillar of liberal epistemology, humanism, and democracy. The scientific enterprise, built on merit, has proven effective in generating scientific and technological advances, reducing suffering, narrowing social gaps, and improving the quality of life globally. This perspective documents the ongoing attempts to undermine the core principles of liberal epistemology and to replace merit with non-scientific, politically motivated criteria. We explain the philosophical origins of this conflict, document the intrusion of ideology into our scientific institutions, discuss the perils of abandoning merit, and offer an alternative, human-centered approach to address existing social inequalities.
Great work! The only problem? See where the paper was published (after being rejected elsewhere).
Emergent Ventures Africa and Caribbean, third cohort
Dr. Keabetswe Ncube is a Geneticist from South Africa. Her EV grant is for her work in using statistical and genetic inferences to help rural farmers maximize yields.
Frida Andalu is a petroleum engineer by training from Tanzania and a Ph.D. candidate. Her EV grant is to assist in her research of developing plant-based volatile corrosion inhibitors to mitigate top-of-line corrosion in natural gas pipelines.
Desta Gebeyehu is a biochemical researcher from Kenya. Her EV grant is to assist in her research of developing bioethanol-gel fuel from organic waste.
Bobson Rugambwa is a software engineer from Rwanda. After graduating with a master’s from Carnegie Mellon University he co-founded MVend to tackle the problem of financial inclusion in Rwanda.
Sylvia Mutinda is a Chemist and Ph.D. researcher from Kenya. Her EV grant is to assist with her search on strigolactone biosynthesis focusing on countering striga parasites in sorghum farms in Kenya.
Dr. Lamin Sonko, born in the Gambia and raised in the U.S., is an Emergency Medicine physician and recent Wharton MBA graduate. He is the founder of Diaspora Health, an asynchronous telemedicine platform focused on patients in the Gambia and Senegal.
Cynthia Umuhire is an astronomer from Rwanda and Ph.D. researcher. She works as a space science analyst at the Rwanda Space Agency. Her EV grant is to assist her in establishing a knowledge hub for junior African researchers in space science.
Brian Kaaya is a social entrepreneur from Uganda. He is the founder of Rural Solars Uganda, a social enterprise enabling rural households in Uganda to access electricity through affordable solar panels.
Shem Best is a designer and urban planning enthusiast from Barbados. His EV grant is to start a blog and podcast on urban planning in the Caribbean to spur discourse on the built environment in the Caribbean and its impact on regional integration.
Susan Ling is an undergraduate researcher from Canada. Her EV grant is to continue her research on biodegradable, long-acting contraceptive implants with a focus on Africa, and general career development
Elizabeth Mutua is a computer scientist and Ph.D. researcher from Kenya. Her EV grant is to assist in her research on an efficient deep learning system with the capacity to diagnose retinopathy of prematurity disease.
Youhana Nassif is the founder and director of Animatex, the biggest animation festival in Cairo, Egypt. His EV grant is for the expansion of the festival and general career development.
Esther Matendo is a Ph.D. candidate in food science from the Democratic Republic of the Congo. Her EV grant is to assist in her research on plant-based treatments of mycotoxin contamination on maize in South Kivu (one of the main maize production zones in the DRC).
Alex Kyabarongo is a recent graduate of veterinary medicine from Uganda. He is now a political affairs intern at the Implementation Support Unit of the Biological Weapons Convention at the United Nations Office for Disarmament Affairs in Geneva. His EV grant is for general career development.
Margaret Murage is a Ph.D. researcher from Kenya. Her EV grant is to assist in her research of developing new photosensitizing agents for photodynamic therapy for cancer treatment.
Kwesiga Pather, for design and development of low-cost drones for agricultural uses in Uganda and general career development.
Dr. Sidy Ndao is a materials engineer by training from Senegal. He is the founder and President of the Dakar American University of Science and Technology (DAUST). The university provides a rigorous American-style English-based engineering education to African students.
Chiamaka Mangut is a Ph.D. candidate at Columbia University from Nigeria. Her EV grant is to fund new field research using archaeobotanical methods to study ancient populations in the Jos Plateau.
Dr. Yabebal Fantaye is Cosmologist by training from Ethiopia. He is the co-founder of 10 Academy, a training bootcamp to assist recent graduates of quant fields to acquire remote data science-related jobs.
For his very good work on these award I wish to heartily than Rasheed Griffith. And here is a link to the previous cohort of Africa winners.
The first recorded scientific grant system?
“Encouragements” from the French Académie des Sciences, 1831-1850.
The earliest recorded grant system was administered by the Paris-based Académie des Sciences following a large estate gift from Baron de Montyon. finding itself constrained in its ability to finance the research of promising but not-well-established savants, the academy seized on the flexiblity afford by the Montyon gift to transform traditional grands prix into “encouragements”: smaller amounts that could broaden the set of active researchers. Even though the process was highly informal (the names of the early recipients were not published in the academy’s Compte rendus), it apparently avoided suspected or actual cases of corruption…Throughout the 19th century, however, the academy struggled to convince wealthy donors to abandon their preference for indivisible, large monetary prizes in favor of these divisible encouragements.
That is from the Pierre Azoulay and Danielle Li essay “Scientific Grant Funding,” in the new and highly useful NBER volume Innovation and Public Policy, edited by Austan Goolsbee and Benjamin F. Jones. (But according to the book’s own theories, shouldn’t the book be cheaper than that?)
Executive in Residence, Math Talent Search
Who We Are: Carina Initiatives
Carina Initiatives is a philanthropic fund working to send more kids from more communities to the frontiers of science and technology. We see math as fundamental to future innovation; as such, we fund and support organizations that work to inspire, unearth, and train math talent.
Here is the link. This has the promise to be an important post, but it needs the right person.
A Mosquito Factory?!
A “mosquito factory” might sound like the last thing you’d ever want, but Brazil is constructing a facility capable of producing five billion mosquitoes annually. The twist? The factory will breed mosquitoes carrying a special bacteria that significantly reduces their ability to transmit viruses. As far as I can tell, however, the new mosquitoes still suck your blood.
Nature: The bacterium Wolbachia pipientis naturally infects about half of all insect species. Aedes aegypti mosquitoes, which transmit dengue, Zika, chikungunya and other viruses, don’t normally carry the bacterium, however. O’Neill and his colleagues developed the WMP mosquitoes after discovering that A. aegypti infected with Wolbachia are much less likely to spread disease. The bacterium outcompetes the viruses that the insect is carrying.
When the modified mosquitoes are released into areas infested with wild A. aegypti, they slowly spread the bacteria to the wild mosquito population.
Several studies have demonstrated the insects’ success. The most comprehensive one, a randomized, controlled trial in Yogyakarta, Indonesia, showed that the technology could reduce the incidence of dengue by 77%1, and was met with enthusiasm by epidemiologists.
In Brazil, where the modified mosquitoes have so far been tested in five cities, results have been more modest. In Niterói, the intervention was associated with a 69% decrease of dengue cases2. In Rio de Janeiro, the reduction was 38%3.
Wolbachia-infected mosquitoes have already been approved by Brazilian regulatory agencies. But the technology has not yet been officially endorsed by the World Health Organization (WHO), which could be an obstacle to its use in other countries. The WHO’s Vector Control Advisory Group has been evaluating the modified mosquitoes, and a discussion about the technology is on the agenda for the group’s next meeting later this month.
Do older economists write differently?
The scholarly impact of academic research matters for academic promotions, influence, relevance to public policy, and others. Focusing on writing style in top-level professional journals, we examine how it changes with age, and how stylistic differences and age affect impact. As top-level scholars age, their writing style increasingly differs from others’. The impact (measured by citations) of each contribution decreases, due to the direct effect of age and the much smaller indirect effects through style. Non-native English-speakers write in different styles from others, in ways that reduce the impact of their research. Nobel laureates’ scholarly writing evinces less certainty about the conclusions of their research than that of other highly productive scholars.
Here is the full NBER paper by Lea-Rachel and Daniel S. Hamermesh.
Strong and Weak Link Problems and the Value of Peer Review
Adam Mastroianni’s has an excellent post on strong-link vs weak-link problems in science. He writes:
Weak-link problems are problems where the overall quality depends on how good the worst stuff is. You fix weak-link problems by making the weakest links stronger, or by eliminating them entirely.
Food safety is a weak link problem, bank or computer security is a weak-link problem, many production processes are weak-link, also called O-ring problems.
[But] some problems are strong-link problems: overall quality depends on how good the best stuff is, and the bad stuff barely matters….Venture capital is a strong-link problem: it’s fine to invest in a bunch of startups that go bust as long as one of them goes to a billion.
….Here’s the crazy thing: most people treat science like it’s a weak-link problem.
Peer reviewing publications and grant proposals, for example, is a massive weak-link intervention. We spend ~15,000 collective years of effort every year trying to prevent bad research from being published. We force scientists to spend huge chunks of time filling out grant applications—most of which will be unsuccessful—because we want to make sure we aren’t wasting our money.
These policies, like all forms of gatekeeping, are potentially terrific solutions for weak-link problems because they can stamp out the worst research. But they’re terrible solutions for strong-link problems because they can stamp out the best research, too. Reviewers are less likely to greenlight papers and grants if they’re novel, risky, or interdisciplinary. When you’re trying to solve a strong-link problem, this is like swallowing a big lump of kryptonite.
At Maximum Progress, Max Tabarrok has some nice diagrams illustrating the issue:
If you have a weak-link view of science, you’d think peer review works something like this. The relationship between quality and eventual impact is linear, or perhaps even bowed out a bit. Moving resources from low input quality projects to average ones is at least as important to eventual impact as moving resources from average projects to high quality ones.
In a strong-link model of science, filtering the bottom half of the quality distribution is less important to final impact [because the impact of research is highly non-linear].
Even though peer review has the same perfect filter on the quality distribution, it doesn’t translate into large changes in the impact distribution. Lots of resources are still being given to projects with very low impact. Although the average input quality increases by the same amount as in the weak link model, the average final impact barely changes. Since peer review has significant costs, the slightly higher average impact might fail to make up for the losses in total output compared to no peer review.
This is a simplified model but many of the simplifying assumptions are favorable for peer review. For example, peer review here is modeled as a filter on the bottom end of the quality distribution…But if peer review also cuts out some projects on the top end, its increase of the average impact of scientific research would be muted or even reversed.
Eight Things to Know about LLMS
A good overview from computer scientist Samuel R. Bowman of NYU, currently at Anthropic:
1. LLMs predictably get more capable with increasing investment, even without targeted innovation.
2. Many important LLM behaviors emerge unpredictably as a byproduct of increasing investment.
3. LLMs often appear to learn and use representations of the outside world.
4. There are no reliable techniques for steering the behavior of LLMs.
5. Experts are not yet able to interpret the inner workings of LLMs.
6. Human performance on a task isn’t an upper bound on LLM performance.
7. LLMs need not express the values of their creators nor the values encoded in web text.
8. Brief interactions with LLMs are often misleading.
Bowman doesn’t put it this way but there are two ways of framing AI risk. The first perspective envisions an alien superintelligence that annihilates the world. The second perspective is that humans will use AIs before their capabilities, weaknesses and failure modes are well understood. Framed in the latter way, it seems inevitable that we are going to have problems. The crux of the dilemma is that AI capability is increasing faster than our AI understanding. Thus AIs will be widely used long before they are widely understood. You don’t have to believe in “foom” to worry that capability and control are rapidly diverging. More generally, AIs are a tail risk technology, and historically, we have not been good at managing tail risks.
What do we need to talk to whales?
We detail a scientific roadmap for advancing the understanding of communication of whales that can be built further upon as a template to decipher other forms of animal and non-human communication. Sperm whales, with their highly developed neuroanatomical features, cognitive abilities, social structures, and discrete click-based encoding make for an excellent model for advanced tools that can be applied to other animals in the future. We outline the key elements required for the collection and processing of massive datasets, detecting basic communication units and language-like higher-level structures, and validating models through interactive playback experiments. The technological capabilities developed by such an undertaking hold potential for cross-applications in broader communities investigating non-human communication and behavioral research.
That is from a new research paper by Jacob Andreas, et.al., and the (ungated) article offers considerable detail on exactly how to do this. They already have funding from both Dalio and Audacious.
From the comments, on AI safety
This is from Richard Ngo, who works on the governance team at OpenAI:
A few points:
1. I agree that the alignment community has generally been remiss in not trying hard enough to clarify the arguments in more formal papers.
2. The only peer-reviewed paper making the case for AI risk that I know of is: https://onlinelibrary.wiley.com/doi/10.1002/aaai.12064. Though note that my paper (the second you linked) is currently under review at a top ML conference.
3. I don’t think that a formal model would shed much light here. My goal in writing my paper was to establish misaligned power-seeking AGI as a credible scientific hypothesis; I think that most who think it’s credible would then agree that investigating it further should be a key priority, whether or not their credences are more like 10% or more like 90%.
From this batch of comments. Here is Richard on Twitter.
This GPT-4 answer speaks for itself
TC: Which is the leading attempt to publish a canonical paper on AGI risk, in a leading science journal, refereed of course. The paper should have a formal model or calibration of some sort, working toward the conclusion of showing that the relevant risk is actually fairly high. Is there any such thing? Please answer as would a leading scientific expert in the relevant fields, including computer science and possibly also population ecology, systems theory, and other possible areas.
GPT-4: As a leading scientific expert, I must clarify that while there are numerous papers discussing various aspects of AGI (Artificial General Intelligence) risk, no single paper can be labeled as the “canonical” paper on the subject. It is worth noting that the field of AGI risk is interdisciplinary, involving computer science, artificial intelligence, ethics, population ecology, systems theory, and other domains. Consequently, the body of research on AGI risk is diverse, with various approaches and models, some of which may involve formal calibration.
One paper that may be of interest to you is “The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation” by Brundage et al., published in 2018. This paper, co-authored by experts from various institutions, including OpenAI, the University of Oxford, and the Center for a New American Security, discusses potential risks from the malicious use of AI and offers recommendations for researchers, policymakers, and industry practitioners. While this paper does not provide a formal calibration model, it does shed light on the potential risks associated with AGI.
Another important work in the field is Nick Bostrom’s book “Superintelligence: Paths, Dangers, Strategies,” which discusses the risks associated with the development of AGI and offers possible strategies to mitigate those risks. Bostrom’s work has been influential in shaping the discourse on AGI risk, but it is not a refereed journal article….[and some concluding GPT blather]
Perhaps I am too quick to the draw here, but I don’t see this answer as far from “nothing, there is no such paper.” Comments are open, of course, if you can surpass GPT-4, which perhaps is better at evolving into a world-destroying entity than it is at finding citations. Further prods did not change the basic answer, and if anything GPT models tend to confabulate or hallucinate entries, not deny them. Or perhaps in this case it is hiding the refereed articles and deceiving us?
And maybe I’ve missed it, but I’ve also never seen Scott Alexander or Zvi point to such a paper, or even a good example of a rejected paper aiming in this direction. Nor have I seen them make a big stink about the absence of such a paper, though in virtually any other area they will hit you with a fire hose of citations and links to published models in referred journals.
I’ve also asked a whole bunch of “people who ought to know” and not received a single concrete answer, one such individual responding immediately with the answer “zero.”
In part, I would like to encourage those fascinated with AGI risk to try to create and publish such a paper, or perhaps to fund it or otherwise encourage it. Something more systematically fleshed out than “10 reasons why lists of 10 reasons might be a winning strategy.” It would go a long way to giving the idea more credibility in the scientific community, not to mention with yours truly. How about Nature? Science? Somewhere else? I know top journals can be closed or unfair, but at the very least you can put the paper and the associated referee reports on-line for the rest of us to judge. And then try it in a lesser journal, it still will get traction and you will get valuable feedback, of a very different kind than from on-line forums.
If the chance of existential risk from AGI is 99 percent, or 80 percent, or even 30 percent, surely some kind of modeled demonstration of the basic mechanics and interlocking pieces is possible. Indeed a certain kind of clarity should be evident, at least conditional on the more extreme views being correct. In general, I am not a fan of the “you should work on this!’ strategy, but if you think the whole future of the entire world is at stake…shouldn’t you be obsessed with working on such a thing, if only to convince the rest of us? And in as many different formats as possible, including the methods most commonly recognized by the scientific community?
In the meantime, if you are a young person interested in this issue, and you observe such a paucity of refereed, published model-based papers in the area — consider any area just to get your mind off the fraught and emotional topic of AGI existential risk — what would you infer from that absence?
And what if said community of commentators almost universally insisted they were the most extreme of rationalists?
Now none of this means the claims about extreme risk are wrong. But you can think of it as a kind of propaedeutic to reading the literature and current debates.
Addendum: I have looked at papers such as these:
https://arxiv.org/abs/2206.13353, https://arxiv.org/abs/2209.00626, https://arxiv.org/abs/2109.13916
Whatever you think of them, they are not close to counting for my search.
The Nuclear Non-proliferation Treaty and existential AGI risk
The Nuclear Non-Proliferation Treaty, activated in 1970, has been relatively successful in limiting nuclear proliferation. When it comes to nuclear weapons, it is hard to find good news, but the treaty has acted as one deterrent of many to nation-states acquiring nuclear arms. Of course the treaty works, in large part, because the United States (working with allies) has lots of nuclear weapons, a powerful non-nuclear military, de facto control of SWIFT, and so on. We strongly encourage nations not to go acquiring nuclear weapons — just look at the current sanctions on Iran, noting the policy does not always succeed.
One approach to AI risk is to treat it like nuclear weapons and also their delivery systems. Let the United States get a lead, and then hope the U.S. can (in conjunction with others) enforce “OK enough” norms on the rest of the world.
Another approach to AI risk is to try to enforce a collusive agreement amongst all nations not to proceed with AI development, at least along certain dimensions, or perhaps altogether.
The first of these two options seems obviously better to me. But I am not here to argue that point, at least not today. Conditional on accepting the superiority of the first approach, all the arguments for AI safety are arguments for AI continuationism. (And no, this doesn’t mean building a nuclear submarine without securing the hatch doors.) At least for the United States. In fact I do support a six-month AI pause — for China. Yemen too.
It is a common mode of presentation in AGI circles to present wordy, swirling tomes of multiple concerns about AI risk. If some outside party cannot sufficiently assuage all of those concerns, the writer is left with the intuition that so much is at stake, indeed the very survival of the world, and so we need to “play it safe,” and thus they are lead to measures such as AI pauses and moratoriums.
But that is a non sequitur. The stronger the safety concerns, the stronger the arguments for the “America First” approach. Because that is the better way of managing the risk. Or if somehow you think it is not, that is the main argument you must make and persuade us of.
(Scott Alexander has a new post “Most technologies aren’t races,” but he doesn’t either choose one of the two approaches listed above, nor does he outline a third alternative. Fine if you don’t want to call them “races,” you still have to choose. As a side point, once you consider delivery systems, nuclear weapons are less of a yes/no thing than he suggests. And this postulated take is a view that nobody holds, nor did we practice it with nuclear weapons: “But also, we can’t worry about alignment, because that would be an unacceptable delay when we need to “win” the AI “race”.” On the terminology, Rohit is on target. Furthermore, good points from Erusian. And this claim of Scott’s shows how far apart we are in how we consider institutional and also physical and experimental constraints: “In a fast takeoff, it could be that you go to sleep with China six months ahead of the US, and wake up the next morning with China having fusion, nanotech, and starships.”)
Addendum:
As a side note, if the real issue in the safety debate is “America First” vs. “collusive international agreement to halt development,” who are the actual experts? It is not in general “the AI experts,” rather it is people with experience in and study of:
1. Game theory and collective action
2. International agreements and international relations
3. National security issues and understanding of how government works
4. History, and so on.
There is a striking tendency, amongst AI experts, EA types, AGI writers, and “rationalists” to think they are the experts in this debate. But they are only on some issues, and many of those issues (“new technologies can be quite risky”) are not so contested. And because these individuals do not frame the problem properly, they are doing relatively little to consult what the actual “all things considered” experts think.
*A New History of Greek Mathematics*
I have read only about 30 pp. so far, but this is clearly one of the best science books I have read, ever. It is clear, always to the point, conceptual, connects advances in math to the broader history, explains the math, and full of interesting detail. By Reviel Netz. Here is a brief excerpt:
And this is how mathematics first emerges in the historical record: the simple, clever games accompanying the education of bureaucrats.
One of the best books of the year, highly recommended.
My excellent Conversation with Jessica Wade
Here is the audio, video, and transcript. Here is part of the summary:
She joined Tyler to discuss if there are any useful gender stereotypes in science, distinguishing between productive and unproductive ways to encourage women in science, whether science Twitter is biased toward men, how AI will affect gender participation gaps, how Wikipedia should be improved, how she judges the effectiveness of her Wikipedia articles, how she’d improve science funding, her work on chiral materials and its near-term applications, whether writing a kid’s science book should be rewarded in academia, what she learned spending a year studying art in Florence, what she’ll do next, and more.
Here is the opening bit:
COWEN: Let’s start with women in science. We will get to your research, but your writings — why is it that women in history were so successful in astronomy so early on, compared to other fields?
WADE: Oh, that’s such a hard question [laughs] and a fascinating one. When you look back at who was allowed to be a scientist in the past, at which type of woman was allowed to be a scientist, you were probably quite wealthy, and you either had a husband who was a scientist or a father who was a scientist. And you were probably allowed to interact with science at home, potentially in things like polishing the lenses that you might use on a telescope, or something like that.
Caroline Herschel was quite big on polishing the lenses that Herschel used to go out and look at and identify comets, and was so successful in identifying these comets that she wanted to publish herself and really struggled, as a woman, to be allowed to do that at the end of the 1800s, beginning of the 1900s. I think, actually, it was just that possibility to be able to access and do that science from home, to be able to set up in your beautiful dark-sky environment without the bright lights of a city and do it alongside your quite successful husband or father.
After astronomy, women got quite big in crystallography. There were a few absolutely incredible women crystallographers throughout the 1900s. Dorothy Hodgkin, Kathleen Lonsdale, Rosalind Franklin — people who really made that science possible. That was because they were provided entry into that, and the way that they were taught at school facilitated doing that kind of research. I find it fascinating they were allowed, but if only we’d had more, you could imagine what could have happened.
COWEN: So, household production you think is the key variable, plus the ability to be helped or trained by a father or husband?
The discussion of chirality and her science work is very interesting, though hard to summarize. I very much like this part, when I asked her about her most successful unusual work habit:
But just writing the [Wikipedia] biography of the person I was going to work with meant that I was really prepped for going. And if I’m about to see someone speak, writing their biography before means I get this. That’s definitely my best work habit — write the Wikipedia page of what it is that you are working on.
I don’t agree with her on the environment/genes issue, but overall a very good CWT, with multiple distinct parts.