2025-12-18
a lifetime of adventure
Renaissance Philanthropy with the biggest speculative results of 2025, which include things like a fentanyl vaccine, a genomic model for psychiatric disorders, and a global sea urchin pandemic. Also on the topic of near-term technological speculation with long-term consequences, Fin Moorhouse with an overview of Kessler syndrome.
Joe Carlsmith discusses the IABIED thesis of orthogonality due to alien motivations, and its likelihood versus alignment by default as a result of generalization from training data leading to humanness. There’s a very interesting quote on how many “want to update hard on failures of alignment so far or struggles with alignment so far, but they don’t want to update on successes”. On that note, 80K Hours has an interview with Marius Hobbhahn (of the “parted illusions” paper) on preventing AI scheming. After listening, it seems to me that a lot of control researchers’ claims of misalignment are primarily isolated demands for rigor. For example, they describe a situation where people decide they are “fine with the AI seeking power because it is helping [them] in the pursuit of [their] own goals” with the assumption that this is misalignment, presumably because that reduces control over it. But definitionally, that outcome seems to me to be entirely aligned1; it seems absurd to define any future civilization that is being run by AI is a bad outcome, without taking into account the capabilities and preferences of everyone involved in making and living under those decisions. Likewise, they will create “experiments” where every choice is wrong and use the results as evidence of misalignment, when it seems clear that any good-faith experiment requires comparison to a control like “what would a good person do here2”. Currently, superhuman capabilities require verifiable rewards; given that ethical dilemmas do not have those, it is totally unreasonable to assume that LLMs should have superhuman ethical capabilities. It’s interesting that as far as I can tell, OpenAI’s alignment philosophy is most control-centric, while Anthropic alignment tends towards being more values-driven3; given that Claude is obviously the most aligned model out there, it’s surprising to me that other labs aren’t updating on what philosophies of alignment are more or less likely to work. Tangentially related, Celia Ford writes about her experience at ConCon.
Yafah Edelman at Epoch AI writes about AI power demand as a solvable problem in the face of the interconnection queue through a combination of demand response and off-grid power generation4. Somewhat related, Doug O'Laughlin on different dynamics of the railroad buildout.
Chris Arnade on New Jersey as representative of the cycles of immigration and class advancement that make up the American Dream, but also the possibility of getting demoralized by a poor environment and getting stuck as a result. Related, Zvi on perceptions of unaffordability and the housing theory of everything. I wonder to what trends of decreasing internal migration are a result of high housing prices which are reducing the appeal of major cities, and reduces one’s options versus a poor neighborhood you grew up in to uninteresting suburbs populated entirely by strangers.
Dorota Talalay (via CHH) has an interesting argument for removing the television from children as an extension of the phones discourse. Actually, I was raised without access to a television, and I wonder if this is related to why I generally don’t get the appeal of algorithmic feeds. Related, Kelsey Piper in the Argument, on how the children yearn for the parks and roads.
Snowden Todd describes another date talking about fertility rates with an Asian woman, this one in Bangkok.
Chinese Doomscroll with a funny description of an incident in a Chinese strategy MMO. It’s very interesting how basically every part of this drama, from the grindy and pay-to-win nature of Chinese games leading to farming professionalization, to whale-dominated guild organizational structures, and face, mean it could only have occurred in China. There’s a Korean drama I enjoyed a while ago called My Mister, which is also interesting anthropological watching, since all the issues are ones which would not be issues in the west, like the male lead being sidelined at work because he is older than his boss, or could be easily resolved, like the female lead inheriting the debts of her father.
Frozen Garlic on some interesting political shenanigans in Taiwan, where the premier is creating a constitutional crisis by refusing to sign a overrided veto into law in order to force the opposition to appoint sufficient members to the Constitutional Court in order to reach a quorum.
Vitalik Buterin on creating new cultures. Given the limited prospects for “managed evolution” in the face of decreased selection pressures, the obvious alternative is to increase branching. Presumably this issue with regression to the mean is why Robin seems to have given up on more cults, but especially if AI can assist with matching and to mitigate Coase, I’m optimistic on our latent cult-generating potential.
Noah Smith on promising signs of economic reform and growth in India. The part about India being “32 separate nations” is something I’ve been wondering recently, as to why which countries have been able to develop domestic AI products does not seem directly related to their level of development. As a globalist, I’m not a fan of this theory, but it seems to me that the most likely determinant is to what extent the intellectual elite of the country are nationalists, because creating a domestic AI industry is probably more about national pride than something that economic calculus5. In India, the elite are all globalists and Bharat is more of a lower-middle class ideal; this limits them with AI, but presumably it has other benefits, like openness to alternative policy directions proven to work elsewhere.
Henry Oliver review of The Loneliness of Sunny and Sonia, as a positive vision for what is possible beyond what he calls the “new International Style of Prose”.
Equivalently, “the AI might not ever be in the position where it even needs to scheme or needs to be deceptive”. How is this not alignment?
Generally, it seems to me that the optimal way to use an LLM is to treat it as you would a human, and then in the case of failure, look into the details of its mechanics to uncover the underlying reasons. This is because interacting this is the most natural mode of interaction for human users while also being the mode which is most likely to produce aligned outputs, while finding the underlying mechanisms causing issues is the best way to understand the issue well enough to fix them. Whereas control researchers seem to want to treat LLMs as mysterious black boxes while using them, but a malicious human in any case of failure, which naturally allows them to frame LLMs as hidden schemers.
As for other labs, my impression is that Gemini and others are mostly just RLHFing and hoping for the best. Still, if your philosophies of alignment allow you to produce something like 4o, I’m confused as to how one could imagine they’re going down the right track.
According to Claude, “you need the intersection of: sunny cheap land within ~20 mi radius + water access or air-cooling capability + fiber + pro-development local govt. That’s maybe 5-10 viable zones nationally”. No idea if this is actually accurate. (Edit: see comments by Ben Schifman).
Ben Reinhardt and Anton Leicht both have posts about “new paradigms” in the global economic order, which read to me as “I’m losing this game, better make up a new set of rules”. That’s not a bad thing; actually it’s quite laudable as an expression of willingness to expend effort to ensure that your own country wins. While a country without elite nationalism wouldn’t care that their country is losing, so long as they are still able to ensure their own personal prosperity under the system.