I also find it hypocritical when the copyright industry fails to put any effort into prosecuting these big techs for their so called infringements.
It's like the industry is a shadow of its former self. The way the copyright industry used to operate, one would think these big tech CEOs would wake up with SWAT pointing guns at them while their electronics are seized, and then they'd end up in court and get hit with something ridiculous like a quadrillion dollar fine.
It actually pisses me off that it's not happening. Not because I care about copyright, but because it's extremely disrespectful towards all the previous victims of the copyright industry.
China is already going as hard as they can on their own GPUs. When has availability of non-Chinese tech in China meant China didn't ultimately come up with homegrown replacements?
It's just my general impression. They banned China from the ISS, China made their own space station. China's making their own x86 chips, their own GPUs.
As a fellow wheel reinventer, I admire their audacity. It's the sort of thing that makes me wish my country was like China.
Classic case of getting downvoted solely because it's a negative comparison to China. The letter for letter exact same comment in a thread only about the US on here, that doesn't mention China, would get upvoted.
You are, of course, right. All of the downsides with none of the benefits.
You might want to compare Singapore with a city like NYC or London, not with a territorial state. It's pretty normal around the world for cities to be replenished mainly by people moving in.
(Of course, to be fair you then also need to compare GDP per capita against other cities. And they usually do a lot better than territorial countries that include a lot of hinterland.)
Not only x86 chips, they are going in fairly hard on Risc V and Loongarch (MIPS/Risc V inspired ISA). Risc V is still growing trying to catch up to ARM, while Loongarch LA664/LA864 chips are much closer to x86 performance than other options. They still are many years behind but not as far as you would expect.
GPU's are still a fair way behind with Moore Threads S80 being a better example of their high end. I suspect they have some major driver issues because they current benchmark far below what that silicon should be able to do. https://en.mthreads.com/product/S80
There is also the pressure to have them innovate on older process nodes so they can make this stuff domestically. For instance Huawei is doing what they call 'logic folding' which is basically just stacking dies in a way that ends up reducing the overall size of chip features. Not sure how it addresses thermals but it is a cool idea.
They greatly increased efforts when the US restricted high end exports to them. Unless further restrictions accomplish something worthwhile in the short term they seem unlikely to be of benefit to the US.
They were calling for bans on open weight models. Bans on their competitors. Bans on anyone not as "enlightened" as them.
It is absolutely hilarious that they were the first to get regulated, and that it got to the point they had to turn off Fable as though it had been banned even for american citizens.
So this hinges on a reading of SB 1047 that interpreted the full shutdown requirement as impossible for an open-weight LLM. But it looks like that was already addressed. Here's an analysis:
>Clarifying the scope of a “full shutdown.” SB 1047’s “full shutdown” requirement has been a source of constant consternation for the open-source community. CalChamber explains:
>Under SB 1047, developers must build “full shutdown” capabilities into their models and may be held liable for downstream uses over which they have no control, impeding their ability to open-source their models. Ultimately, liability should rest with the user who intended to do harm, as opposed to automatically defaulting to the developer who could not foresee, let alone block, any and all conceivable uses of a model that might do harm. While recent amendments seemingly seek to narrow what is meant by “full shutdown” capabilities, the exclusions are unnecessarily difficult to interpret as drafted (full shutdown “does not mean the cessation of operation of a covered model to which access was granted pursuant to a license that was not created by the licensor…”) and altogether insufficient.
>Committee amendments simplify and clarify the definition of “full shutdown” such that the shutdown capability can be implemented into hardware used to train or run a model, rather than the model itself. The amendments also serve to exclude covered model derivatives that are outside of the developer’s control.
> may be held liable for downstream uses over which they have no control
Equivalent to a ban. Nobody is going to host or invest in this stuff if they suddenly become liable for everything it does. This is equivalent to repealing the safe harbor provisions in the DMCA.
Are there any indications that this will be possible? Consumer hardware will continue getting better but I can't see 512GB RAM in a MacBook Pro any time soon. I'm hoping linear attention techniques plus MoE will make breakthroughs in size/compression and throughput.
Well, we're probably not going to be running frontier models anytime soon, but I think the general assumption is smaller models will continue to improve until they're sufficiently good frontier models aren't needed.
There's potentially also augmentation through tools, harnesses and RAG to help boost how well they work without tons of parameters.
so around US$150k which is Small/Medium-Enterprise territory already, but who knows when it will hit "reasonable" home consumer territory
I think there's hope future generations of unified memory machines may get this sort of memory availability when new fabs open in then next couple of years and then ramp up production for a few years afterwards - that makes ~2030s credible at this point, but nobody can really predict the market that far ahead
> I think there's hope future generations of unified memory machines may get this sort of memory availability
I hope you're right. This is a very exciting idea. The weights are out there. The demand is astronomical. The manufacturers just need to make it happen.
there are cheaper ways to do it. not like, consumer-cheap, but I'm setting up a rig for 80% cheaper than that.
I'm a tad worried about triggering a run on the particular hardware I'm buying though so I'll leave it vague here, but hit me up on Discord if you're curious.
You're correct, yet it's also a fact that Anthropic was attempting regulatory capture in order to limit open weight models, cripple their competition and solidify their market position. Nothing wrong with enjoying the sheer Schadenfreude of the situation. Their self-serving fearmongering had the most hilariously unexpected result possible.
Maybe. But I disagree in general with "nothing wrong with enjoying the sheer schadenfreude of the situation". Schadenfreude is a description of a common human impulse that is not a good impulse!
Normally I would agree with you, but in this case I'm gonna make an exception. These are billionaires who campaign against open weight models. Their misfortunes are our freedom.
I also find it hypocritical when the copyright industry fails to put any effort into prosecuting these big techs for their so called infringements.
It's like the industry is a shadow of its former self. The way the copyright industry used to operate, one would think these big tech CEOs would wake up with SWAT pointing guns at them while their electronics are seized, and then they'd end up in court and get hit with something ridiculous like a quadrillion dollar fine.
It actually pisses me off that it's not happening. Not because I care about copyright, but because it's extremely disrespectful towards all the previous victims of the copyright industry.
reply