2025-10-28
words
Thinking Machines has open-sourced a more efficient way to run distillation on LLMs1, which is sort of analogous to letting the chess student see the eval bar after all of their moves. This reminds me that when Deepseek R1 was released, one of the major debates was to what extent their capabilities were obtained through distillation of frontier models through API calls. On that note, Institute for Progress on the possibility that the US might sell Blackwell chips to China in the current round of trade negotiations. But if distillation is this good, compute might not really a bottleneck anymore to reaching frontier capabilities2.
Timothy Lee with some graphs on AI investment and usage trends. It’s actually pretty useful for understanding exactly why Altman is positive on ads all of a sudden, because it seems increasingly unlikely to me that OpenAI’s current product mix and trajectory can successfully blitzscale a significant portion of their rapidly growing userbase to paying customers. It’s interesting that we seem to be replicating similar dynamics, where B2B SaaS can run by subscriptions (Anthropic and Gemini), but consumer tech still relies on ads.
Kevin Munger comments on the Knausgård essay about how words and the humanities present a mediated view of physical reality. Possibly related, GLADIA Research Lab suggests that LLMs are invertible.
David Deek investigates the fertility rates of different Japanese municipalities to extract potentially effective pronatalist policies.
Fred Gao translates an interesting talk by Fu Peng on how Chinese demographic shift will affect national consumption habits.
Cartoons Hate Her on sharing restaurant food. Personally, I think sharing food is obviously superior, and as a result I find dietary restrictions and pickiness annoying. Possibly as a result, I think sharing food should be more of a norm, in order to stigmatize food pickiness.
Adam Mastroianni on increasing sameness everywhere. I feel like he answered his own question, in that this is entirely about different phenomena related to scale. Regarding production, this is obtained through abstraction frameworks which enable it: websites all look the same because everyone is using Material UI and Tailwind; restaurants are all the same because of Sysco and Virtual Dining Concepts; physical products are constrained by automation. Likewise, personality shifts are a result of increasing urbanization, which reinforces increasing bureaucratization. Presumably weird stuff is still being made, it just doesn’t have reach or last very long, because it doesn’t make any money.
Brian Potter on the rise and fall of British shipbuilding capabilities.
Nintil linkthread.
Lauren Gilbert linkthread.
Edit: even better modification from Huggingface.
That being said, it might not be an entirely a terrible thing, because while it means the performance gap essentially disappears, it also puts China in a local optima which ensures that they won’t pull ahead either.
It’s rather ironic because last year Gwern described a hypothesis that a combination of chain-of-thought and distillation would allow us to reach recursive self-improvement, because you could train a big model, distill it into a light model to run chain-of-thought, and then loop those inputs into the next big model. This is almost what we’re getting here, in that the frontier labs train the top models, then the Chinese labs distill them into open-source models along with performance improvements, and then everyone uses them to run their experiments.
Anyway, from the perspective of Thinking Machines, presumably they’re commoditizing their complements, because the world where everyone runs their own customized RL is one where the open-source models are both lightweight and cutting edge. But it’s bad for the frontier labs to be turned into commodities, so it will be interesting to see how they end up responding.
I treat restaurants similarly to how I treat any art. I go because I want to try new things, or create a new reference point for something I’ve already tried, and I want to experience as fully as I can what a restaurant has to offer and be able to judge it after. This means trying a lot of things, so sharing is the obvious choice. If I’m going to a restaurant with other people, I expect them to be there with a similar level of interest (or at least curiosity), and the food would inspire a somewhat substantial chunk of the conversation.
If I’m going out primarily for the purpose of meeting people rather than for the food, I prefer a basic cafe optimized more for good seating and a convenient location, and I generally pregame on food and just get a beverage for the sake of ordering something, or if I’m hungry I don’t care that much about sharing because it’s established that we aren’t there for the food, which is on the boring side anyway.
I do regularly meet people who just want a big plate of something they like, and I can try to respect that approach, but I also feel like why go out at all if you aren’t driven primarily by curiosity? There’s variance here, but going out is often some combination of expensive, unhealthy, and time consuming. And why go out with the people you’re out with if you aren’t interested in sharing? I think it’s odd to have a social activity where the socializing and the activity are mutually exclusive, and many restaurants are poorly designed for socializing.
But I take food as an interest more seriously than most people and don’t really enjoy going out for the sake of going out.
Once on a drive to a restaurant with my family the topic of sharing came up and I explained why sharing is better: “Suppose every dish we get costs $10. That means we value each dish at at least $10. But most of the value in a dish is in the first handful of bites, and I frequently would be willing to pay the same price for a smaller portion of a dish. I probably value 20% of the dish at $7, and the rest of the dish at $3. Suppose with five of us we order five dishes and none of us share. Then we’d be spending $50 total but each person only gets a $10 value out of the meal. Considering the meal takes up a few hours of our time, this is a poor value. But if we all share each dish and we all value 20% of a dish at $7, then each person is getting a $35 value from the meal. The total remains $50, but the collective value of the meal is $175. This is a 250% consumer surplus!”
I expected some pushback on a few points but the response was surprisingly hostile: “This is a really unpleasant conversation.” “Yeah Sam, that wasn’t nice of you to say.” “Why can’t you just be normal.” “I’m going to order and eat what I want and that’s the end of the conversation.”
(I reject the food communist label, I believe in growing the pie rather than slicing it up! They're the communists!)
I’ll concede there’s a limit to the benefits of sharing, and the part in the article about her in-laws expecting to share very small plates is an example of oversharing to the extent that you lose any of the surplus value. Intelligent sharing involves choosing restaurants and ordering appropriately based on the number of people.
As a predominantly non-picky eater who more or less likes everything that fits within dietary parameters (vegetarian, plus a preference for eating healthy), I find pickiness to be annoying too, and I live with the somewhat arrogant belief that people who have aversions to specific foods are wrong to live their lives that way and could/should be educated out of it. But I can’t relate to them and am unsure if that’s really true. I've thought about this a lot but haven't looked into research on it, so I'll pose as mature and not advocate for stigmatizing the picky eaters quite yet.