I was talking to someone about this the other day. I was part of a team at NASA that developed a cooling system for the ISS and this whole premise makes no sense to me.
1. Getting things to space is incredibly expensive
2. Ingress/egress are almost always a major bottleneck - how is bandwidth cheaper in space?
3. Chips must be “Rad-hard” - that is do more error correcting from ionizing radiation - there were entire teams at NASA dedicated to special hardware for this.
4. Gravity and atmospheric pressure actually do wonders for easy cooling. Heat is not dissipated in space like we are all used to and you must burn additional energy trying to move the heat generated away from source.
5. Energy production will be cheaper from earth due to mass manufacturing of necessary components in energy systems - space energy systems need novel technology where economies of scale are lost.
Would love for someone to make the case for why it actually makes total sense, because it’s really hard to see for me!
1. Solving cost of launching mass has been the entire premise of SpaceX since day one and they have the track record.
2. Ingress/egress aren't at all bottlenecks for inferencing. The bytes you get before you max out a context window are trivial, especially after compression. If you're thinking about latency, chat latencies are already quite high and there's going to be plenty of non-latency sensitive workloads in future (think coding agents left running for hours on their own inside sandboxes).
3. This could be an issue, but inferencing can be tolerant to errors as it's already non-deterministic and models can 'recover' from bad tokens if there aren't too many of them. If you do immersion cooling then the coolant will protect the chips from radiation as well.
4. There is probably plenty of scope to optimize space radiators. It was never a priority until now and is "just" an engineering problem.
5. What mass manufacture? Energy production for AI datacenters is currently bottlenecked on Siemens and others refusing to ramp up production of combined cycle gas turbines. They're converting old jet engines into power plants to work around this bottleneck. Ground solar is simply not being considered by anyone in the industry because even at AI spending levels they can't store enough power in batteries to ride out the night or low power cloudy days. That's not an issue in space where the huge amount of Chinese PV overproduction can be used 24/7.
> There is probably plenty of scope to optimize space radiators. It was never a priority until now and is "just" an engineering problem.
It's a physics problem, as others pointed out, but even if we take it as another "just an engineering problem", have a look at the Hyperloop. Which is similarly just a long vacuum tube, and inside is like an air hockey table, not that big a deal, right?...
That bit reminded me of someone who wanted us to design a patch the size of a small postage stamp, at most 0.2mm thick, so you could stick on products. It was to deliver power for two years of operation, run an LTE modem, a GNSS receiver, an MCU, temperature and humidity sensor and would cost $0.10. And it would send back telemetry twice per day.
The conversation went something like this (from memory):
- We can't do that
- Why not?
- Well, physics for one.
- What do you mean?
- Well, at the very least we need to be able to emit enough RF-energy for a mobile base station to be able to detect it and allow itself to be convinced it is seeing valid signaling.
- Yes?
- The battery technology that fits within your constraints doesn't exist. Nevermind the electronics or antenna.
- Can't you do something creative? We heard you were clever.
I distinctly remember that last line. But I can't remember what my response was. It was probably something along the lines of "if I were that clever I'd be at home polishing my Nobel medal in physics".
Even the sales guy who dragged me into this meeting couldn't keep it together. He spent the whole one hour drive back to the office muttering "can't you do something creative" and then laughing hysterically.
I think the solution they went for was irreversible freeze and moisture indication stickers. Which was what I suggested they go for in the first 5 minutes of the meeting since that a) solved their problem, and b) is on the market, and c) can be had for the price point in bulk.
I like your sales guy. Might have punched them after a while but that's right up there with the time someone tried to tell me there was no iron in steel because it wasn't in the ingredients list. And this someone sold stamped steel parts!
That's so hilarious. I've had a couple that went in that direction but nothing to come close.
To be fair though, there is a lot of tech that to me seems like complete magic and yet it exists. SDR for instance, still has me baffled. Who ever thought you'd simply digitize the antenna signal and call it a day, hardware wise, the rest is just math, after all.
When you get used to enough miracles like that without actually understanding any of it and suddenly the impossible might just sound reasonable.
> Can't you do something creative? We heard you were clever.
The purely digital neighborhood of the SDRs is much easier to explain than the analog rat droppings between the DAC/ADC and the antenna. That part belongs to dark wizards with costly instruments that draw unsettling polar plots, and whose only consistent output is a request for even pricier gear from companies whose names sound an awful lot like European folk duos.
The digital end of SDRs are simple. Sample it, then once you have trapped the signal in digital form beat the signal into submission with the stick labeled "linear algebra".
(Nevermind that the math may be demanding. Math books are nowhere near as scary as the Sacred Texts Of The Dark Wizards)
"Rohde & Schwarz — live at the VNA, 96 dB dynamic range, one night only."
> whose names sound an awful lot like European folk duos.
That had me laughing out loud, you should have left the name out to make it more of a puzzler :)
I apparently have been drawn to the occult for a long time and feel more comfortable with coils, capacitors and transmission lines than I do with the math behind them. Of course it's great to be able to just say 'ridiculously steep bandpass filter here' and expect it to work but I know that building that same thing out of discrete components - even if the same math describes it - would run into various very real limitations soon.
And here I am on a budget SDR speccing a 10 Hz bandfilter and it just works. I know there must be some downside to this but for the life of me I can't find it.
(reference to a character in the Expiditionary Force series by Craig Alanson
Only a very small portion of his physical presence is in local spacetime, with the rest in higher spacetime. He can expand his physical presence from the size of an oil drum or shrink to the size of a lipstick tube. He can’t maintain that for long without risking catastrophic effects. If he did, he would lose containment, fully materialize in local spacetime and occupy local space equal to one quarter the size of Paradise. The resulting explosion would eventually be seen in the Andromeda Galaxy.)
What makes you so sure? SpaceX already has thousands of 6 kW networking racks flying around in LEO and they dissipate their heat just fine, and are plenty cost-effective. You think they can't do any better than that with a new design specifically optimized for computing rather than networking?
Probably, but they likely can't do better than we can do on Earth. Networking in space offers specific advantages that are not easy to replicate on Earth. Data centers in space don't have clear advantages beyond easily debunked ideas about cooling and power.
I'm not talking about the whole idea, just the heat dissipation part. So many people in this thread seem so sure this is impossible because you can't radiate heat in space, completely ignorant to the fact that SpaceX is already dissipating over 20 MW of solar power in LEO in a reasonably cost-effective manner.
The advantage of 24/7 solar power is clear, obvious, and undeniable, it's just a question of whether that's outweighed by the other disadvantages.
The solar panels on the newest satellites can deliver 6kW but the power that satellite actually uses is less. The satellite is only using 300W[1] during the dark phase of it's orbit when it can use it's entire mass to cool down. Is that limit because of the battery or is it because the satellite needs to radiate all the heat it acquired from the other half of the time in the sun?
Looks like that's a purely speculative assumption the blog author made, not a fact. I'm not sure why he made that assumption given that Starlink doesn't actually stop working at night.
Fair point that in SSO you'd need 2-3x the radiator area (and half the solar panels, and minimal/no batteries). I don't think that invalidates my point though.
3. There are WAY more things to get corrupted on a computer system than tokens. And non-determinism does NOT mean it’s tolerant to faults. Random values are intentionally introduced at the right moment for LLMs.
Sounds a bit like that Dilbert where the marketing guy has sold a new invisible computer and is telling the engineers to now do their job and actually make it.
The question isn't "can you mitigate the problems to some extent?", it's "can you see a path to making satellite data centers more appealing than terrestrial?"
The answer is a flat out "no," and none of your statements contradict this.
Terrestrial will always be better:
1. Reducing the cost of launches is great, but it will never be as cheap as zero launches.
2. Radio transmissions have equally high bandwidth from Earth, but fiber is a better network backbone in almost every way.
3. Radiation events don't only cause unpredictable data errors, they can also cause circuit latch-ups and cascade into system failure. Error-free operation is still better in any case. Earth's magenetosphere and atmosphere give you radiation shielding for free, rad-hard chips will always cost more than standard (do they even exist for this application?), and extra shielding will always cost more than no shielding.
4. On Earth you can use conduction, convection, AND radiation for cooling. Space only gets you marginally more effective radiation.
5. Solar is cheaper on the ground than in space. The increase in solar collection capability per unit area in space doesn't offset the cost of launch: you can get 20kW of terrestrial solar collection for around the price of a single 1U satellite launch, and that solar production can be used on upgraded equipment in the future. Any solar you put on a satellite gets decommissioned when the inference hardware is obsolete.
And this ignores other issues like hardware upgrades, troubleshooting, repairs, and recycling that are essentially impossible in space, but are trivial on the ground.
I have no expertise is this area, so I'm not getting into whether or not this idea makes sense.
That being said, this statement strikes me as missing the point:
> Solving cost of launching mass has been the entire premise of SpaceX since day one and they have the track record.
As I understand it, SpaceX has a good track record of putting things into space more cost effectively than other organisations that put things into space.
That is not the benchmark here.
It doesn't matter if Musk can run thousands of data centres in space more cost effectively than (for example) NASA could. It matters whether he can do it more cost effectively than running them on earth.
I don’t think that statement was missing the point. As you point out, what matters is the total cost of ownership of the system. The cost of launching mass into space today isn’t the only reason terrestrial data centers are more cost effective today, but it’s the main one. If you make it cheap enough to send giant solar arrays and radiators to space, the other costs of operating in space may start to look like a small price to pay to eliminate the need for inputs like land and batteries.
> Would love for someone to make the case for why it actually makes total sense, because it’s really hard to see for me!
Elon musk has a history of making improbable-sounding promises (buy a tesla now, by 2018 it will be a self-driving robotaxi earning money while you sleep, humanoid robots, hyperloops).
The majority of these promises have sounded cool enough to enough people that the stock associated with him (TSLA) has made people literal millionaires just by holding onto the stock, and more and more people have bought in and thus have a financial interest in Musk's ventures being seen in a good light (since TSLA stock does not go up or down based on tesla's performance, it goes up or down based on the vibes of elon musk. It is not a car company stock, it is an elon vibes check).
The thing he's saying now pattern matches to be pretty similar, and so given Musk's goal is to gain money, and he gains money by TSLA and SpaceX stock going up, this makes perfect sense as a thing to say and even make minor motions towards in order to make him richer.
People will support it too since it pattern matches with the thing prior TSLA holders got rich off of, and so people will want to keep the musk vibes high so that their own $tsla holdings go to the moon.
The story here is even simpler. SpaceX is going public this year. Elon made a monumentally shitty investment in Twitter and then poured a stupid amount of money into xAI at the peak of the cycle. By having SpaceX buy xAI, he gets to swap worthless shares in that company for more SpaceX liquidity. Simple as that.
Exactly, and there needs to be some economic justification for a giant rocket. There is no money to be made by going to Mars, and AI data centers in space could attract investors (who are just riding the data center hype).
> I data centers in space could attract investors (who are just riding the data center hype).
I find this to be the most obvious game plan here. Makes total sense from financial engineering point of view.
You _might_ get to develop nice tech/IP to enable other space based businesses at the same time. "we sold them on X but delivered Y". So it's a bit of a hail mary, but makes total sense to me if you want to have a large budget for inventing the future.
Once you can demonstrate even a fraction of this capability of operations ... I think you can sell a "space dominance" offering to Pentagon for example and just keep pedaling.
"We are going to build the perfect weapon" does not necessarily entice as large engineer population as "we are going to Star Trek".
Another thing - if Moon is going to be a thing, then _properties on Moon_ are going to be a thing.
In theories of value in post-ai societies scarce assets like land are going to become more valuable. So it's a long term plan that makes sense if you believe Moon will be a realestate market.
Ah, this old fallacy. There are myriad examples of the rich striving to be richer and the powerful fighting to gain even more power. Why would it be any different with Musk? If anything I suspect (this is absolutely an unverifiable opinion; I am not stating it as fact) that Musk's driving force is to become the first trillionaire.
Billionaire money is not like money for the normal person. It is a placeholder for how much influence you have on the economy - and even the state.
It is not just a number, as it is for people who just save a few dollars, for whom it really is just a number until they withdraw money to use it. The billionaire's money is not "money", it is actual working assets, and the abstraction of turning this into a number does a terrible job, the result now misunderstood by many. Assets being companies doing stuff mostly (holding non-control-giving paper assets is different and not what being a top capitalist is about, only used as an additional tool below the actual goal). Which they fully control (the small investor does not even have any control worth mentioning when they own shares of a public company).
They don't just play with money, they play with real things! And they want to play with ever bigger real things. They don't just want to improve some minor product. They want to control the fate of civilization.
OT:
I hate this money view with a passion, this is what too many people discussing wealth inequality issues get wrong. This is not Scrooge McDuck and his money pile. Money is an abstraction, and it is misused terribly, hiding what is actually going on for too many observers who then go on to discuss "numbers".
That is also why the idea to "just redistribute the money of the rich" is a failure. It isn't money! It is actual real complex organizations. And you can't just make everything into a public company, and also, even when they are, for better or worse owners don't lead like managers. Doing the socialism thing (I grew up in the GDR) where everybody owns a tiny bit of everything just does not work the same.
We will have to look at what those super-rich are actually doing, case by individual case of ownership, not just look at some abstract numbers. Sometimes concentrated control over a lot of assets is a good thing, and other times it is not. Ignoring the objection of "who would control that?", because right now they control themselves so it's never nobody.
I think you have read the "redistrubute the money" people wrong. They definitely, absolutely want to reduce the power the tiny minority hold over the many. That's the whole point. The money is a tool to get the work done.
What’s silly is a bunch of so-called intelligent entrepreneurs and tech insiders twisting themselves into a pretzel coming up with reasons why this or that won’t work, by the guy who keeps doing the impossible
What exactly has Elon done that's "impossible"? Like the Boring Company where he promised 1,000x faster boring? It turned into a mile or two of a poorly routed hole with some Teslas tossed down into it. He and his shills hand waved away the problem, confident their brilliance would allow them to dig 1,000x faster than modern commercial boring. It never happened.
The only impossible thing Elon has done is make fantasy claims and real people fall for it.
I will definitely credit Elon with building a company that made reusable self-landing rockets seem routine and boring. That was definitely "impossible".
Pretty much everything else though is just vapourware.
It was impossible in the sense that nobody else did it before. It was not impossible as in you need to violate basic laws of Physics or elementary Economics.
Before reusable rockets, the idea made sense. Building a rocket is expensive; if we reuse we don’t have to keep spending that money. Fundamentally, rockets are rockets. It’s not like they invented anti-gravity or anything.
It’s like climbing the Everest. Before it was done, it was still something people could plan and prepare for. But you’re not going to climb all the way to the moon, even with oxygen bottles. It’s a completely different problem to solve.
The most difficult point to argue against for people who want to defend Musk’s delusions is simple economics: at the end of the day, when you’ve solved
- the energy source problem (difficult but probably doable);
- the radiation-resistant chips issue (we know we can do it, but the resulting chip is not going be anywhere near as fast as normal GPUs on Earth);
- the head dissipation problem (physically implausible, to be charitable, with current GPUs, but considering that a space-GPU would have a fraction of the power, it would just be very difficult);
- the satellite-to-satellite communication issue, because you cannot put the equivalent of a rack on a satellite, so you’d need communication to be more useful than a couple of GeForces (sure, lasers, but then that’s additional moving parts, it’s probably doable but still a bit of work);
- the logistics to send 1 million satellites (LOL is all I can say, that’s a fair number of orders of magnitude larger than what we can do, and a hell of a lot of energy to do it);
- and all the other tiny details, such as materials and logistics just to build the thing.
Then, you still end up with something which is orders of magnitude worse and orders of magnitude more expensive than what we can already do today on Earth. There is no upside.
Yeah, but landing a rocket backwards also sounded very improbable to me, yet it looks pretty cool now.
Also people made fun of tesla that it will never be able to compete with the big carmakers. Now I would rather have some stocks in tesla than holding on to volkswagen.
I wouldn’t be so sure about Tesla stock. Tesla has only weathered 1 market downturn cycle and that was when it was a very different company. The company has thus far had access to plentiful capital since the Model S started being delivered.
Famous investors like to repeat the quote that “when the tide goes out, that’s when we find out who’s wearing no pants.” When Tesla actually weathers its first market downturn is when we find out how much investors interest is maintained When investment dollars are scarce.
Regarding 3, they're almost certainly thinking of putting these in SSO where the radiation environment isn't too much worse than you see on the ISS. If they were going to go outside the Van Allen belts it would be a different story.
The whole AI bubble makes way more sense if you model it not as rational economic activity, but rather as the actions of a rogue AI slowly taking over our institutions and redirecting them towards its goals. Data centers in space make no sense economically, but think of how survivable the rogue AI will be once we build those orbiting data centers! (I am joking about this, but it's weird that my logic makes sense.)
for the chips to both be radiation hardened and as powerful as our current chips they'd need to be massive. There's a reason the mars rover uses a PowerPC G3
>Ingress/egress are almost always a major bottleneck - how is bandwidth cheaper in space?
Free space optics are much faster than data to/from the ground. If the training workloads only require high bandwidth between sats, this isn’t a real issue.
> Chips must be “Rad-hard” - that is do more error correcting from ionizing radiation - there were entire teams at NASA dedicated to special hardware for this.
They don't do RAD hardening on chips these days, they just accept error and use redundant CPUs.
There are apparently rad-hard DDR4 chips these days so this is patently false. SpaceX used to talk a lot about substituting rad-hard components with triple redundant regular x86 years ago, that's true.
I think I've also seen someone mention that the cost and power benefit of substituting rad-hard chips with garden variety wean off fast once the level of redundancy goes up, and also it can't handle deep space radiations that just kill Earthbound chips rather than partially glitching them.
You are confidently incorrect. Even Starlink uses rad-hardened CPUs. Redundant error correction is only really an option on launch hardware that only spends minutes in space.
Note that on modern hardware cosmic rays permanently disable circuits, not mere bitflips.
No, he's not. Dragon is using CotS, non rad-hardened CPUs. And it's rated to carry humans to space.
> AWST: So, NASA does not require SpaceX to use radiation-hardened computer systems on the Dragon?
John Muratore: No, as a matter of fact NASA doesn't require it on their own systems, either. I spent 30 years at NASA and in the Air Force doing this kind of work. My last job was chief engineer of the shuttle program at NASA, and before that as shuttle flight director. I managed flight programs and built the mission control center that we use there today.
On the space station, some areas are using rad-hardened parts and other parts use COTS parts. Most of the control of the space station occurs through laptop computers which are not radiation hardened.
> Q: So, these flight computers on Dragon – there are three on board, and that's for redundancy?
A: There are actually six computers. They operate in pairs, so there are three computer units, each of which have two computers checking on each other. The reason we have three is when operating in proximity of ISS, we have to always have two computer strings voting on something on critical actions. We have three so we can tolerate a failure and still have two voting on each other. And that has nothing to do with radiation, that has to do with ensuring that we're safe when we're flying our vehicle in the proximity of the space station.
I went into the lab earlier today, and we have 18 different processing units with computers in them. We have three main computers, but 18 units that have a computer of some kind, and all of them are triple computers – everything is three processors. So we have like 54 processors on the spacecraft. It's a highly distributed design and very fault-tolerant and very robust.
> Dragon is using CotS, non rad-hardened CPUs. And it's rated to carry humans to space.
Those are not independent facts. They put the hardware inside, behind the radiation shielding they use to keep the astronauts safe. It's why regular old IBM laptops work on the Space Station too. That kind of shielding is going to blow your mass budget if you use it on these satellites.
SpaceX, which prefers COTS components when it can use them, still went with AMD Versal chips for Starlink. Because that kind of high performance, small process node hardware doesn't last long in space otherwise (phone SoC-based cubesats in LEO never lasted more than a year, and often only a month or so).
Which is exactly how you'd do a hypothetical dc in space. Come on, you're arguing for the sake of arguing. CotS works. This is not an issue.
> That kind of shielding is going to blow your mass budget
SpX is already leading in upmass by a large margin. Starship improves mass to orbit. Again, this is a "solved" issue.
There are other problems in building space DCs. Rad hardening is not one of them. AI training is so fault tolerant already that this was never an issue.
None of the discussed designs include radiation shielding like that. Nobody is considering doing it that way, because the math really really doesn’t work out (instead of unshielded, where it just doesn’t work out).
A cosmic ray striking a chip doesn’t cause a bitflip - it blows out the whole compute unit and permanently disables it. It is more like a hand grenade going off.
Between fp nondeterminism, fp arithmetic, async gradient updates, cuda nondeterminism, random network issues, random nodes failing and so on, bitflip is the last of your concerns. SGD is very robust on noise. That's why it works with such noisy data, pipelines, compute and so on. Come on! This thread is having people find the most weird hills to die on, while being completely off base.
Dragon spends 6mo+ in orbit regularly. I have no idea what's happening in this thread, but it seems everyone is going insane. People don't even know what they're talking about, but they keep on bringing bad arguments. I'm out.
... hooked up to the ISS, with humans in attendance to fix anything that goes wrong... not doing very much.
It's akin to the difference between a boat moored up in a port, and an autonomous drone in the middle of the Pacific. Aside from that, satellites have to maneuver in orbit (to stay in the correct orbit, and increasingly to avoid other satellites). Hefting around additional kgs of shielding makes that more difficult, and costly in terms of propellant, which is very important for the lifetime of a satellite.
When they talk about "space" they are, right now, talking about the moon. Which has some gravity. They are putting the data centers on the moon. And the satellites are lunar satellites not earth-orbit satellites. Lonestar physical data center payload landed on the moon in Feb 2025 and Sidus space developing the lunar satellites.
Yeah, the cost of doing it on the moon would be even more astronomical. Then there also is the three second of round-trip latency to consider (ca. 2.6 s just the signal, but processing adds a bit more).
1. Getting things to space is incredibly expensive
2. Ingress/egress are almost always a major bottleneck - how is bandwidth cheaper in space?
3. Chips must be “Rad-hard” - that is do more error correcting from ionizing radiation - there were entire teams at NASA dedicated to special hardware for this.
4. Gravity and atmospheric pressure actually do wonders for easy cooling. Heat is not dissipated in space like we are all used to and you must burn additional energy trying to move the heat generated away from source.
5. Energy production will be cheaper from earth due to mass manufacturing of necessary components in energy systems - space energy systems need novel technology where economies of scale are lost.
Would love for someone to make the case for why it actually makes total sense, because it’s really hard to see for me!