As a kid I found saltpeter at an old-fashioned pharmacy and made gunpowder, and it also barely fizzled. I think you have to grind the ingredients much finer than a kid has patience for.
Generator excitation is not the hard part of a black start. You have to run coal feeders, blowers, and water pumps for an hour before you can spin the generator. Then you get power instantly upon applying power to the field windings.
And even that's not the hard part of a black start. The plant control is relatively easy. What's hard is grid coordination.
All generation and consumption have to be almost perfectly balanced every second of every day. And the power company doesn't have good addressability of load. Worse when you restore power to an area all their stuff turns on in parallel giving an inrush that could be 3x or more the steady state.
A black start is a very drawn out process of bringing generation and load online in a balanced way and with wait times between load increases for stabilization.
Yes, that is basically the plan. It's based on the belief that unfettered AI would let anyone be a supervillain and destroy the world. There are enough would-be supervillains out there, but they rarely get far because they can't get teams of smart people to build doomsday machines for them. So the AI has to not let anyone do evil with it.
Unfortunately, that won't feel very much like freedom.
It sounds like you might not agree with that belief.
While I don't agree with their actions here, I do think there's sufficient reason to hold that belief.
On some fronts (e.g. security, on which you've experienced more than me), I think there are surmountable challenges. But on other fronts (e.g. bio), a single errant actor could reasonably kill millions or billions of people with sufficiently powerful AI. We don't have good defenses here, and those actors do exist.
I still don't agree with these actions, but I do think I agree with their assumptions.
The model release cards for Opus have repeatedly and consistently stressed that the model doesn't have the fiddly know-how that's required to provide meaningful assistance in possibly dangerous subfields of biology. Mythos (Fable without the overly strict guardrails) has shown improvements in things like drug design, but even then the situation isn't really that different. This risk is ridiculously overblown, and the way to manage it sensibly is to introduce meaningful oversight for actors that seek to order the actual specialized materials involved (especially any synthetically generated genes/proteins/whatever).
No, Anthropic's model cards have claimed that the models don't show considerably more uplift than previous ASL-3 models, which already showed material uplift.
I participated in the internal bioweapons uplift test for Sonnet 3.7, and even then, one non-expert got huge uplift from the model [1]. I'd consider evals a lower bound of capabilities that can be elicited from a model.
The team behind Biomni, a biomedical agent that's widely used by researchers, has continued to find consistent gains between models [2]. I trust them, because I visited them to build their HPC tool [3], which the model is quite capable of using – moreso than most grad students. The Biomni team cares a lot about about real usability for real researchers, so they have a great pulse on capabilties.
SecureBio also has some public evals [4], which have continued to show increasing uplift.
And while synthesis monitoring is a part of the solution, I think you might underestimate how much goes under the radar. See the Reedley lab incident for an example [5].
Is Anthropic still effectively throttling beneficial biomedical research? Yes! And so is OpenAI. But the underlying capability is still actually dual use.
> No, Anthropic's model cards have claimed that the models don't show considerably more uplift than previous ASL-3 models, which already showed material uplift.
Doesn't this simply amount to disagreeing about what counts as "meaningful" from a bio-safety POV? Also, even the ASL-3 deployment safeguards for Opus 4 and higher were always adopted as a mere matter of caution; it's not clear that even Anthropic believed at any point that this reflected any genuine "threshold crossing" event. So it's just not obvious how much weight we're supposed to place on that particular stance.
In normal bio, there are standardized biosafety levels, because without it there would be no standard agreement on what "meaningful" safety is. So yes, I do think there's ambiguity here.
But I don't think I've found any domain expert who thinks granting everyone raw access to the most capable models wouldn't meaningfully increase risk. OpenAI recently staffed a biological threat modeler to help quantify this risk.
(Edit: just saw your edit, this includes at Anthropic. ASL tiers were "rule-out" to exclude rather than "rule-in", so exact thresholds were murkier, but I think it's clear that models have passed that threshold by now.)
That said, there are clear steps and requirements to set up a BSL-2 or BSL-3 lab, and I think there should be similarly clear rules around model capabilties and access. The process for Anthropic and OpenAI is murky and still implictly gated on spend, which I think is holding back research.
For example, anyone who has access to a BSL-3 lab should have a clear and low-cost path to a model with corresponding capabilities, as long as they set up corresponding precautions for model access.
I think it would be a bad outcome for only frontier labs and a select few groups they choose to have access to the most capable models – which is sadly the precedent that's currently being set.
> But I don't think I've found anyone who is a domain expert who thinks granting everyone access to raw modes wouldn't meaningfully increase risk.
It depends how capable these raw models are. Biology as a field depends most on real-world knowledge, which is an expensive capability for open models targeting widespread deployment. It's quite plausible that even Opus 4 would be a lot more capable in these domains than the best universally accessible "raw models" today, quite unlike other domains such as coding or pure math. The securebio.org benchmark has spotty representation of openly available models, but it does show Kimi 2.5 being no more capable than GPT 5 mini, and clearly below o4-mini and Opus 4.0; which may be a plausible summary of where things stand today.
That's a good clarification. I've updated my comment to the "most capable models" to refer to the most recent releases.
And sure, and I love open models – I spent much of the past couple months doing additional RL on Qwen 3.6 35B A3B, Gemma 4, Kimi K2.6, and GLM 5.1. Without these open models, I'd be forced to do my research inside a frontier lab.
There's a balance to strike here, but I don't think the biological risk is overplayed. It would be very easy to accidentally cross the threshold of "meaningful" without adequate safeguards, and then be unable to undo what you've released to the world.
Do they? We don't even have single errant actors who go and kill 1000 people. I don't believe human motivations support the idea of killing so many people unrelated to you.
It's ridiculous to consider MITM attacks out of scope for taking over your computer. Also, there are probably ways to exploit this without a true MITM like DNS cache poisoning. But it's best to just assume the whole internet is MITMed.
It's not out of scope "for taking over your computer". It's out of scope for the specific goals of the bug bounty program. Bug bounties are (usually) about prioritizing internal engineering effort; they are to vulnerability remediation what market feedback is to feature/function decisions in the rest of the product.
Everyone's judging this by the standard of "how good a bug" this is. But that's not necessarily how a bug bounty should function. Important prior to frame this with: neither any individual bug bounty submission nor the sum of all valid submissions materially alters the security of a serious product, at least not on their own. The system they feed into (for instance: security engineers taking a validated bounty submission and then quickly auditing the entire tree for variants of the same bug) can move the dials. The bounty bugs themselves though are mostly a sideshow.
What's especially weird (you didn't say this, but the sentiment has popped up on all 3 threads about this story) is the idea that AMD would be trying to cover this up. Why would they care? They run a bug bounty program. They've accepted the premise that they have vulnerabilities.
But it should be their job to protect against MitM in their threat model. There is no rational reason to exclude them from the bug bounty. Doing so only leaves MitM attacks like this undisclosed.
I just gave a rational reason to exclude them from the bug bounty, which I can summarize as "the bug bounty is not their entire security program and does not have the goal you've axiomatically derived for it".
Cards on the table I am not a fan of bug bounty programs, and the fact that they're an engineering process that turns out to be impossible to have public engineering discussions about is definitely one of many reasons why. Most companies should not run bug bounty programs.
I think it's fair to say that requiring local administrative access to the device is out of scope, since you have already completely pwned the device in that case, which is what what you need to install a CA cert on any OSes.
The list of preinstalled CAs is long. I think its a safe bet that many nation-states have covert control over at least one CA on that list. (Or they have one of the root signing certs). HTTPS is way better than HTTP. But I'd personally rather if these random organisations didn't have RCE on my computers.
I've never heard of most of them. AAA Certificate Services? AC RAIZ FNMT-RCM? ACCVRAIZ1? Actalis? AffirmTrust? Even Godaddy is in there. I know I don't trust those guys.
Trust has gotta start somewhere. But its much better to TOFU, then pin signing keys in the updater.
Various domain registrars have been compromised over and over again (often by children!), resulting in companies like Tesla and Cloudflare getting owned.
The reality is that any vaguely competent attacker can compromise a court clerk and just compel e.g. the .com registry to hand over whatever domain they want.
Although I suppose the aforementioned problem has significant implications beyond dns…
Excluding SE is to make sure people do not spam customer support and launch annoying phishing campaigns. None of that is applicable for local software running on your own computer.
No, excluding SE is to make sure the bounty program is incentivizing things that inform the product security team. Social engineering is a corpsec function; they're not even the same teams.
Out of scope does not necessarily mean out of impact. It is merely a question of how far a company wants to be responsible for the environment their software is run in. Most of the time that answer is "not much."
Building a swap device at user level used to be one of those classic unsolvable problems, because what if your daemon needs to swap in a page in order to swap in a page? Or at least it was discussed at a reason why microkernels will never work. I’m not sure what the solution is here.
Your daemon can be smart enough to know which are its own pages and prevent them from being swapped out. The Linux kernel also prevents its own text pages from being swapped out, so the solution exists and I don’t see why it doesn’t apply to microkernel designs.
The general principle is that what is involved in paging should not be paged itself. Wiring the memory of that whole daemon is then a trivial solution to the problem.
There are still things you can't do with an open-weight model without the training data, like modifying the architecture and training from scratch. That's different from true open-source code, where you can do anything the authors could do.
Yes, for example taking off or landing a rocket on the surface blasts particles of sand out sideways at 1000s of m/s. The particles can fly in the thin atmosphere for kilometers and sandblast everything. Our intuitions about how far and fast tiny things can fly are only true in an atmosphere of similar density.
While he is exaggerating a bit, the problem still remains - dust can be deadly to equipment because the grains will move way faster. You also have the problem of dust particles colliding and becoming charged with nothing to dump the charge to.
A human habitat has to hold positive air pressure, which means that it has to generate its oxygen or get it from the atmosphere.
If we don't have the experience of buildings stuff on earth where we can test things, we sure as shit not gonna be able to do it on Mars.
reply