2026-02-12
the black chord
Fin Moorhouse at Forethought has an interesting interview with Sam Hammond on how our institutions of governance might be affected by development and implementation of AGI1. There’s some interesting discussion about the anarcho-capitalist Coasean singularity2, but I’m actually starting to wonder if speculation here is premature, given the current difficulties we have around partner selection. Given that the feedback loops involved are too slow for verifiable rewards, reinforcement learning will have to rely on human feedback in the short term: whether this will succeed then depends on whether the problems we have with hiring are just because of being unable to scale out the methods that work, or a much deeper problem where we never really figured out a general solution in the first place.
Somewhat related to the topic of adoption being forced through by adversarial dynamics, Jasmine Sun describes her visit to Washington DC3 and the anti-AI and anti-data center4 sentiment which she encountered there. I feel like it’s possible to overindex on elite opinion as a representation of the public, since most Americans still trust Amazon and Walmart more than they do their government, and that’s probably not the impression one would get, assuming that the opinions of DC represent what “the public” from outside of Silicon Valley thinks. Still, it occurs to me that one of the reasons that Chinese people tend to have friendlier attitudes towards AI than Americans is due to what production increases implies in a manufacturing versus service-oriented economy: In the former, it means more stuff, while in the latter it mostly just means more competition: if lawyers get cheaper, society might end up spending more than ever before on legal services5; if restaurants start getting automatically booked up by AI agents, then everyone ends up needing to get one. This could become particularly annoying if the transition ends up being blocked by unaccountable laggards at any point in the chain, which could actually produce a net decline in service quality for an extended period of time.
There’s a very interesting philosophical turn at the end of the interview, where Hammond describes his views of cultural relativism, which he claims as being derived from Kant. Which is actually somewhat less crazy than initially seems, since if I understand him correctly, he’s essentially acknowledging Lionel Page’s criticism of the categorical imperative and recovering it by relaxing the requirement of universality. Which means that moral behavior is the socially rational action within the social circles for which it is relevant6. This is essentially used as a basis from which he can discuss potential forms of future governance without getting involved in moral judgments, including his own personal preference for a form which could be described as technofeudalism, where people are more involved in describing preferences and choosing products rather than direct involvement in the machinery of politics and the selection of policy7. On that note, within the framework of gradual disempowerment, there’s an idea that once your labor is no longer economically useful, then your rulers will no longer have any incentive to care about your wellbeing. But it occurs to me that if that is the case, then neither will they have any need for your presence, which means that your right to exit will basically always be preserved. Therefore, the worst outcomes can be mitigated so long as there remains one locality which is still willing to show care, which is why I don’t dismiss the Glen Weyls and Audrey Tangs of the world as being too overly optimistic.
Titotal describes the difficulty of determining which intervention is likely to be the most effective for a given issue. Which is probably less of an issue if you can concievably just try them all.
Niko McCarty on Vibrio natriegens as a potential replacement for Escherichia coli as the microbe of choice for cell cultures if and when AI starts running biological experiments, given their faster replication rate (Edit: more on V. natriegens from same author, and a correction).
Ruxandra Teslo and Derek Lowe comment on the FDA’s refusal to file for the Moderna influenza vaccine (Edit: and Alex Tabarrok).
Scott Alexander on the concept of political backflow. It seems to me that how people view other countries fits into three levels of hierarchy: the first where someone thinks only in terms of their own country, as if all countries were the same as theirs; then another where countries or regional groupings are really just mirrors of their own; which finally splits out with the realization that people of each country are recursively applying your previous level of analysis to their own situations and your own country, prompted by a style of Gell-Mann amnesia when you see what kind of flawed analysis people apply to places which you know just a little more about.
Scott Sumner political economy linkthread.
Ivan Vendrov has an interview with Romeo Stevens which is also on emerging social changes which are taking place alongside or as a result of AI, but from the perspective of an individual agent rather than from a structural societal level.
While on this topic, Alexandre Variengien writes about their initial experiments on using AI agents in negotiation. Ozy Brennan has a cool interview with Shoshannah Tekofsky on the AI Village project, where one of the things they mention is a sort of excessive agreeability and following along, and it seems to me that this really arises as a result of LLMs not having any particular rooted identity. Another reason I think that human-AI merger is the best end-state, but failing that, having each LLM be paired with a human they know they are supposed to serving would probably also be sufficient. Also, not really related but cool, Brian Christian with an interesting paper on LLM self-blinding counterfactuals.
On the topic of the Washington Post, Derek Thompson interviews Jim VandeHei, and Harry Cheadle has a good post on how media companies should be profitable. To some extent, it actually feels like Jim’s joke proposal of a media company focused on things like sports betting tips might actually be the solution to subsidizing deep investigative journalism, since that’s sort of what prediction markets are providing in practice. Then with Hunterbrook to investigate public companies, and ProPublica to investigate the government, it might actually be because others are encroaching into their most prestigious domain that establishment papers are finding it increasingly hard to survive.
I’m confused why data centers are allowed to be so ugly, given the significant overlap between AI companies and the Progress Studies crowd. This is the perfect opportunity to test out whether objections to building aesthetics are actually a major cause of public opposition to construction, or whether it’s really primarily an excuse.
Edit: Justin Curl on this topic and reasons why legal services potentially might not even get cheaper in the first place.
On that note, Maia Mindel has a piece about how we should decouple our sense of worth from our careers, which doesn’t directly mention the threat of AI automation but seems clearly an ambivalent take on that anxiety. In my opinion, she’s totally right, but as an individualist, I don’t quite understand the meaning of this “we”, because it seems to me that this is just something that anyone can just decide for themselves (somewhat related, Cate Hall on how you can just not do things). Personally, I don’t like working, so I stopped, accepting as a result the accompanying reduction in living standards and the hit to status, which is as it should be given how such actions are contrary to the categorial imperative. This is sort of the Goomba fallacy, but it’s strange to me that some people on the left will often use social disapproval as a reason for why they cannot do something, when the left is also the side which is pro-therapy and favors blank slatism (so be the change you want to see). It seems to me that fear of losing life meaning with job loss is really only a reasonable position under the priors of the social conservative right, in which case anti-AI sentiment among white-collar Democrats are even less excusable, seeing as job loss within the blue-collar workforce has historically been dismissed so readily.
Edit: it’s interesting that skepticism of democracy now seems like a bipartisan consensus, at least among the American elite, with the realization and acknowledgement that “populism” really is after all just another way of saying “democracy”. See for example Matt Yglesias on “stealth democracy” as an antidote to increasing perceptions of corruption, and Bryan Caplan telling Donald Wittman “I told you so”. It’s unfortunate because the US could be copying the best aspects of Estonian or Taiwanese e-governance which clearly work, but as yet the necessary infrastructure has not begun to be set up, and the political capital to do so is probably also missing.