2026-01-25
ミス・ブランニュー・デイ
Possessed Machines is the recent AI whitepaper of note1, arguing that AI safety and policy has become captured by structural economic and psychological forces, such that every individual involved has no ability to shape outcomes but is rather co-opted themselves, which means that the field can no longer function as a check against things going badly. But it seems to me that this has always been the case, because after all, we live in a society2. In general, it seems to me that the problem with AI safety is that at the moment, there is not enough safety to do. This might be a controversial claim, given the entire goal of pausing AI is to slow down capabilities so that safety can “catch up”. But what I mean is that the reason that the field of safety can become captured is because everything is run from within the ecosystem, controlled through access to hiring, funding, and connections, thereby funneling everyone through predetermined routes. But if it were truly an open field, then it’s ludicrous that research could be gatekept in such a manner; if there truly is too much left to be done, then any intelligent outsider should be able to pick up an open-source model and in playing around with it, discover some low-hanging fruit inside. That does not appear to be the case, and it seems to me the reason for that is that safety has overtaken capabilities3. Worse, pouring additional funding into the field is merely raising the height of the barriers between those who are either in or out, increasing the strength of selection and the insularity of those inside.
Somewhat related, Judge Glock in Works in Progress4 on how overzealous Environmental Protection Agency mandates on water management without regard to cost-effectiveness are responsible for rising household water prices. Whenever I read something like this, I feel like there should be a clear justification between government spending and fundraising through mechanisms like earmarking. But then I read about various California ballot initiatives, and I’m reminded why this maybe isn’t actually a very good idea.
Tina He on rote memorization as being necessary for creativity. Though it seems to me that a better rule than having 25% memorization would be to introduce random forgetting, which allows you to boost the percentage allocated to habit-driven behavior while also periodically checking if your model needs to be updated (taking advantage of our capacity for continual learning).
Kevin Kodama review of There is No Antimemetics Division, focusing on how much of its literary reception excises its connection to the SCP universe, which ignores its origin as a piece of internet fiction.
ABB on digital IDs as an obviously beneficial public good which most governments are unexpectedly ignoring. Presumably because there is no single department which would be obviously in charge of such a project.
Tyler Cowen on the scenery of New Zealand.
Climbing Gold podcast on preparing for Taipei 101.
Any McKenzie neurobiology linkthread. It feels promising that they seem to be getting a lot longer and more detailed recently. Somewhat related, Evolutionary Psychology Podcast interviews Michael Graziano on beliefs around consciousness. It seems to me that a lot of people like to conclude that machines can or can’t be conscious based on the fact that there is no proof for the opposing position, when of course that’s all the more reason to be uncertain in both directions, and to act in each case according to the principle of least regret. On that note, Bentham’s Bulldog on the potential moral worth of digital minds.
Unclear what it’s relation is to this one “The Shigalyovist Turn”.
There was an episode of Doom Debates recently with Audrey Tang. As I mentioned before, it seems to me that there’s a fundamental difference in understanding between those who understand East Asian political philosophy and those who don’t. For the most part, AI safety researchers seem to be viewing alignment by default with some sort of befuddled delight, and are unable to provide any explanation as to why it is happening besides luck: without understanding the mechanism, they are also unable to reason about its potential costs or limitations. People who see society as not just an abstraction, but as emerging out of individuals organizing and self-aligning from the bottom-up have a different model of the world as to what extent AI alignment as a solved problem, or at least a theoretically solvable one.
Edit: Matthew Barnett comments on this in a recent article.
Via an (unpaywalled!) Brian Potter linkthread. For some reason, Works in Progress has stopped updating their Substack. I’m not sure if this is intentional as a means to promote their print edition. (Edit: posted now).