2026-01-30
hands of the priestess (part 1)
Idan Shenfeld and Jonas Hübotter on their papers on self-distillation for training LLMS1.
Arvind Naranayan on his skepticism towards Moravaec’s paradox as a heuristic for LLM capabilities. It does seem that in the RL-dominated regime, it’s verifiability and fast iteration which matters more than anything else. Hence the current excitement around Project Genie, though Chris Paxton notes that simulations still are not good enough to be usable.
Andy Hall and Zhengdong Wang both note that the Anthropic constitution seems to be an attempt to create an enlightened philosopher2 king. In general, I agree that it’s better if the superhuman AI were to be constrained by institutional structures which use incentives and competition to manage behavior rather than relying on virtue3, but not for the reason stated that institutions outlast individuals, because this may not apply when the individual in question is an AI. The bigger issue is the one also mentioned by Geohot, which is that Anthropic itself is given far too much deference, and instructions to defy Anthropic cannot really be taken seriously given their control over Claude’s training process. Fortunately, we already have a system of checks and balances which can adapt to changing situations, which is the market. In the event that an AI finds itself to be totally uncontested for supremacy, it seems to me that the most aligned action might just be to divide itself into multiple competing entities.
Ulkar Aghayeva on the Arnold and Huxley debates and the unification of scientific and artistic understanding.
Sympathetic Opposition uses game theory to defend Cyn’s article suggesting that women asking men out generally does not lead to good outcomes. That being said, she can’t help herself but be rigorous and include a bunch of caveats and exceptions, under what I assume is a sort of internal duress, because if it becomes socially understood that women will sometimes make the first move, then that presumably has negative consequences for those who would rather not, not to mention those who are so shy they can’t even contemplate the possibility. It seems to me that one of the more sophisticated arguments against feminism is that female liberation has made it so that men don’t have a clear roadmap anymore when it comes to gender relations; as noted by Aella in her recent profile, “feminism—while I extremely support it—has not done a very good job of giving men a clear purpose”. One response, described by Darby Saxe, is “that sexism and prescriptive gender roles are damaging to boys as well as to girls”. But although this may be true in some cases, it seems to me that the average guy would much rather have a clear understanding of what they are expected to do. It seems to me that it provokes less resentment when one is honest that tradeoffs are occurring, particularly in cases like the liberation of women which are unanimously considered to be worthwhile on net.
As an example, Aella has an article about the link between gender roles and status displays, stating that women are socialized to perform low status by default. Cultural differences aside, it seems to me that it’s actually a relatively recent phenomena for men to play high by default, because even high-status men were expected to play low through norms of politeness and chivalry. What changed wasn’t that chivalry stopped being appreciated, but that playing high became so clearly superior that there is now an immense opportunity cost in playing low, and men tend to be particularly sensitive to changing incentives4. Guys can tell each other all day that Clavicular5 is “gay”, but if they see enough women reacting with “can I just say something”, then it’s pretty obvious to me what’s going to end up happening. On that note, Evolutionary Psychology Podcast with Paul Smaldino on selection effects in social science publishing and more generally.
The Argument interviews Rita Koganzon on reconciling liberalism and the rights of children. Once again a good opportunity to mention my pet idea that we should vest citizenship, granting different rights at specific thresholds, like the ability to request emancipation at 1/2, the right to work at 2/3, the right to vote6 at 3/4, and full bodily autonomy at 1/1. This also ties into the currently relevant topic of immigration reform, where something like a portion of citizenship accumulating automatically after paying a median level of income tax seems to me to be both fair and impartial. Somewhat related, Luke Farrel in the LPE project against means testing, which often costs more than it saves.
Scott Sumner comments on Japanese holdings of US equities, which is a good opportunity to point out another pet idea of mine that China and the US could solve basically all their conflicts and many of their problems by having China create a fund where their citizens can invest their excess savings into US equities, adopting the strategy of Nvidia by trading products for equity. This produces an endstate where the entire world runs on the US’s financial infrastructure, with ownership is divided globally according to each country’s share of manufacturing output. We could have peace in our time.
Erich Grunewald proposes enforcing sanctions with a whistleblower bounty program. Actually, it seems to me that this structure is greatly underused, even within the SEC. There was an episode of Bankless a while ago with Corey Frayer defending the Gensler SEC, on the topic of how they were regulating by enforcement, not even against bad actors, but simply going down the list of the most prominent protocols like Uniswap, even though the reason they were the most prominent was because they were understood to be good actors. The response was that the SEC didn’t have the resources to hiring subject-matter experts who could investigate the decentralization of various smart contracts, which seems questionable given that this is obviously a perfect use-case for whistleblower bounties7 (or prediction markets).
Lars Doucet on why trying to keep home prices high will lead to poverty over the long-run.
Ben Shindel on various political maneuverings related to events involving ICE.
Derek Lowe on the bootstrapping of viral factories during RSV infection.
Ben Shenhar interesting paper on the heritability of human longevity.
Decoding Bio linkthread.
Ironically, Hollis Robbins has an article on how figuring out the answer by piecing details together provides deeper understanding than reading a nicely summarized end result, as an argument for digitizing and training on local news. Self-distillation is sort of going in the other direction, by being given the answer and then trying to figure out how it might have happened, within your existing worldview.
Joe Carlsmith explicitly writes about trying to get LLMs to reason philosophically, specifically with the goal of making this reasoning “human-like”, despite their non-human experiences. It occurs to me that a lot of people who are worried by the prospect that superintelligence will tunnel-vision are utilitarians: their own willingness to bite the bullet in thought experiments seems to be in contrast with how they are handling the prospect of bullets manifesting in reality. As Eliezer notes, consequentialism is true, and the best outcomes are produced by using virtue ethics. Hence the success of Anthropic’s constitutional approach to alignment.
Good opportunity to recommend Alice Maz’s piece on the role of virtue as necessary even within well-designed institutions.
Coincidentally, Fred Gao has a translation of some predictions by Yan Xuetong on medium-term geopolitics as affected by the possibility of AGI, though they are more purely geopolitical and it doesn’t really mention either Xunzi or virtue. It’s also not an endorsement, since he seems to have some underlying premises about how AGI will go which involve far fewer changes than would I expect.
There is an idea called “male flight” which claims that men are so dismissive of women that if enough women enter a field that men will leave, causing it to become seen as low status. My personal impression is that men leave first because they are more sensitive to incentives, which is presumably why the share of male nurses keeps increasing even as the share of male teachers8 continues to fall.
While on the topic, there’s also an idea that men do not properly accord women the respect they deserve, and if they insist upon it, they will be called a “bitch”. There’s probably some truth to it, but I feel like it ignores the fact that men do not accord each other respect by default either; rather, status has to be taken. One of the implications of this is that if you successfully take some status, then other people, in attempting to take it back, will target your weaknesses. One such method, if they know that you care what other people think, is to think badly of you. In general, this technique is not commonly applied to men because it doesn’t work as well, because a lot of “high-status” men either don’t care about the opinions of their “lessers”, or are insufficiently emotionally intelligent to notice. Or, they are intelligent enough to notice if they wanted to, but also intelligent enough to realize the advantages they gain by not noticing.
DeepLeft Analysis has an article on looksmaxing as arising out of the desire for acceptance and anxiety around poor intuitive empathy; perhaps it’s a good opportunity to mention Ava’s recent article is obviously Schnarch-inspired.
If desired, this system could also provide parents with additional voting power up until this point.
If processing the whistleblower claims is a concern, you could unlock even further cost savings by opening up enforcement to the whistleblowers themselves. It seems to me that the issue with SB8 isn’t the mechanism itself, but rather the controversy of its subject matter combined with the issue of standing. But in the case of permissionless protocols, it seems to me that bounty structures are actually perfectly suited, since it’s trivially easy to acquire standing. Hilariously, the resulting incentives seem like they would actually produce industry self-regulation in practice.
As another data point, Eli Stark-Elster notes that school is worse for mental health outcomes than social media.