Many parallels between this article and another posted today ("Unsubscribe from the Church of Graphs" https://news.ycombinator.com/item?id=47604253). Both are responses to astralcodexten posts, and the vibes felt around anecdata vs statistics.
We found 500 zero-days in ten year old widely used open-source projects. Was that not a demonstration of the catastrophic failure of human debugging capability?
The Duke harness was specifically designed for these puzzles, that's why they don't want to measure it.
My reading of that part in the technical report (models "could be using their own tools behind the model’s API, which is a blackbox"), is that there's no way to prevent it.
But from fchollet's comment here, using tools and harnesses is encouraged, as long as they are generic and not arc-agi specific. In that case, the models should be benchmarked by prompting through claude code and codex, rather than the through API (as from the api we only expect raw LLM output, and no tool use).
Are you prompting the models through their APIs, which are not designed to use tools or harnesses? Or do the "system prompt" results come from prompting into the applications (i.e. claude code, or codex, or even the web front-ends)?
Astrophysicist David Kipping had a podcast episode a month ago reporting that LLMs are working shockingly well for him, as well as for the faculty at the IAS.[1]
It's curious how different people come to very different conclusions about the usefulness of LLMs.
Textbook models typically simulate normal development of an embryo, e.g. A-P and D-V (anterior-posterior and dorsal-ventral) patterning. The question Levin raises is how a perturbed embryo manages to develop normally, both "picasso tadpoles" where a scrambled face will re-organize into a normal face, and tadpoles with eyes transplanted to their tails, where an optic nerve forms across from the tail to the brain and a functional eye develops.
I haven't thoroughly read all of Levin's papers, so I'm not sure to what extent they specifically address the issue of whether textbook models of morphogen gradients can or cannot account for these experiments. I'd guess that it is difficult to say conclusively. You might have to use one of the software packages for simulating multi-cellular development, regulatory logic, and morphogen gradients/diffusion, if you wanted to argue either "the textbook model can generate this behavior" or that the textbook model cannot.
The simulations/models that I'm familiar with are quite basic, relative to actual biology, e.g. models of drosophila eve stripes are based on a few dozen genes or less. But iiuc, our understanding of larval development and patterning of C Elegans is far behind that of drosophila (the fly embryo starts as a syncytium, unlike worms and vertebrates, which makes fly segmentation easier to follow). I haven't read about Xenopus (the frogs that Levin studies), but I'd guess that we are very far from being able to simulate all the way from embryo to facial development in the normal case, let alone the abnormal picasso and "eye on tail" tadpoles.
I'm not an expert on the actual biological mechanisms, but, it makes intuitive sense to me that both of those effects would occur in the situation you described from simple cells working on gradients: I was one of the authors on this paper during my undergrad[1] and the generalized idea of an eye being placed on a tail and having nerves routed successfully through the body via pheromone gradient is exactly the kind of error I watched occur a dozen times while collecting the population error statistics for this paper. Same thing with the kind of error of a face re-arranging itself. The "ants" in this paper have no communication except chemical gradients similar to the ones talked about with morphogen gradients. I'm not claiming it's a proof of it working that way, ofc, but, even simpler versions of the same mechanism can result in the same kind of behavior and error.
"Some people become depressed at the scale of the universe, because it makes them feel insignificant. Other people are relieved to feel insignificant, which is even worse. But, in any case, those are mistakes. Feeling insignificant because the universe is large has exactly the same logic as feeling inadequate for not being a cow. Or a herd of cows. The universe is not there to overwhelm us; it is our home, and our resource. The bigger the better." -- David Deutsch
I wouldn't attack people's emotions like that, the approach of 'my opinion is better than yours and your emotions are wrong' ain't the best.
Its just one of those concepts or facts of life like our (im)mortality that each of us has to handle on their own terms since each of us is wired in pretty unique ways. Its perfectly fine to be in awe or even stunned by it, it means one actually started to grasp vastness of that topic and the fact we don't have it all figured out and during our lifetime this won't change.
Every time I look at starry night sky and realize those distances, thermonuclear furnaces glowing across vast distances in absolute cold (or their massive groups looking similarly yet being vastly further), I am in awe. It puts my efforts and happiness in my life in a good perspective, in similar fashion spending my time with my kids does. And I look at stars every night I can, its a beautiful calming sight for me.
Emotions are like waves; we can’t choose which ones appear, but we can choose which one to surf. The person you’re replying to (well, really, the person they quoted) didn’t really seem to be “attacking” anyone’s emotions to me. It seemed pretty gently advising the reader not to spend all their time riding the emotional waves that lead to depression or nihilism.
I do believe we all must face these emotions to become aware. But the depression/nihilism trap is very real for many people (myself included), and learning to walk that line and stay curious is part of our emotional/psychological development.
Maybe you saw this paper about that idea - The Genomic Code: The genome instantiates a generative model of the organism.
"Here, we propose a new analogy, inspired by recent work in machine learning and neuroscience: that the genome encodes a generative model of the organism. In this scheme, by analogy with variational autoencoders, the genome does not encode either organismal form or developmental processes directly, but comprises a compressed space of latent variables."
Those papers say the virus is natural and wasn't engineered by humans. The OP link is about the theory that origin event is a natural virus escaping from a lab. A wild-type virus specimen obtained from bats in the wild that was being studied in the lab (not engineered or modified in the lab).
reply