Affordable, mature hardware. The Talos systems are starting at 3k for a quad core CPU on a micro ATX board. Same thing happened to MIPS and Sparc. Performance and technical merit mean nothing vs cheap and ubiquitous hardware. It's a lot of cash for a what amounts to an experimental toy. They are also a bit finicky as a friend bought one from another dev that refuses to post for unknown reasons. So there is risk involved too, no one else is making these boards.
The performance gaps and architectural features that made these chips matter 20 years ago have been closed by commodity off the shelf x86 hardware and various Arm CPU's are eating everything.
The only reason Risc-V matters is that no one has to pay for licenses.
> How about providing a single board computer like a Raspberry Pi
The issue is more about connection to external servers than the particular board itself. A product that both runs great out of the box, and is also hackable to the point where it doesn't 100% rely on the original company being around to run is the sweet spot I'm thinking of. I'm mostly just thinking aloud about what features the product needs to have in order to hit that sweet spot.
> That said, I'm not buying any IoT devices, self-hosted or not, in the foreseeable future
That's cool, but this isn't about the average HN user. Consumers want to, and indeed they do, pay for these devices. I just want a nice model where these products can be sold while not leaving the customer stuck if the original company goes under.
> they're so ashamed of the pile of spaghetti code they're peddling to their customer that they just can't OpenSource it for fear of losing face and reputation for ever.
LOL no. Everyone knows how this particular sausage is made. It's obvious we are talking of an organically growing decades old codebase. It won't surprise anyone.
To be more on topic, ever since I learned of the PicoEVB a few months ago, I am just giddy of the possibilities, it's amazing we finally got a mobile and PCIe connected FPGA. I wasn't even looking for one because I haven't thought it possible. We truly live in the future.
Security and x86? I would maybe not hate this if there were a stipulation that these companies must move beyond x86 with a completely new architecture, something more reasonable for the modern era. Theres too much legacy crud that just muddies the waters when trying to learn how to program these contraptions. Maybe some way to completely disable speculative / out-of-order execution. What are WE getting out of this? By WE I mean non-giant-chip-conglomerate shareholders/welfare queens.
> Couldn't they have fit that within both budget and size?
Oh they could have. The problem is: it's extremely hard to get access to powerful SoCs - the vendors simply won't work with you and most of the documentation is under NDA.
Reminds me of another 'we will make it up in volume' dotcom scam I-Opener https://en.wikipedia.org/wiki/I-Opener selling ~$500 worth of computer and LCD monitor at $99
Using a high volume Amazon chip. Sadly the software stack is not what this thing deserves. It's a full blown DPU used with something very minimalistic SW wise.
>haven't been running their own LLMs for the last five years
Because the hardware has not existed.
This said by accident I've seen hardware that was brought to a testing company by federal marshals that was massively parallel custom hardware that was likely for signal processing a lot of channels at once. So there is plenty of custom hardware out there, but these items have not been produced at the scale needed (from what anyone can tell) and, again from what we can tell, they don't have the general processing capability that GPU/TPU driven LLMs have.
>> They are nowhere near as flexible as a microcontroller is, and reprogramming them for a different purpose is a huge investment from a development standpoint.
Isn't that the point of LiteX and Migen though? To reduce the barrier to entry in time and budget? Before the RPi came along SBC's were fairly expensive, and the RPi was only able to offer a product at that price because of scale.
> VThe problem is with building a competitive stack, Huawei can't get access to a 7nm fab in China
yet
> Huawei can build 7nm chips at TSMC, but due to their ARM l
ARM -> RISC, as ARM is not the only game in town. For example, at the moment I am having a lot of fun with a Unielec u7621 board (2 mini PCIe, mSata and USB3 with hardware accelerated NAT to handle 1Gbps) that costed me about $45 : https://openwrt.org/toh/unielec/u7621-06
I would love to see MIPS64 SOCs with a few GBs of RAM to replace my raspberries
> To be fair, POWER is an open standard, and there's absolutely nothing stopping someone like Linode, DigitalOcean, or Hetzner from offering POWER-based systems at a smaller hourly price.
Well besides the fact that 1) IBM is the only party with both the capabilities and interest in making high performance Power based products [1], and 2) evidently does not understand how to (or why to) invest in bringing this to a general audience. I don't really care if they would do this in IBM Cloud or with other cloud infrastructure companies, but they don't seem to be doing either. In addition 3) why would other parties be interested in running Power when they can run amd64 or arm? It certainly doesn't look to have a price advantage...
IBM really needs to shepherd Power well - they're the only ones that can do it. But I can't help but thinking they seem to be leading it to the grave despite apparently very capable engineering.
[1]: And why would anyone but IBM go for Power at this point if they can have ARM too? An open license matters very little when compared to ARM's mindshare and momentum.
IDGAF about any of that, lol. I just want an API endpoint.
480 tokens/sec at $0.27 per million tokens? Sign me in, I don't care about their hardware, at all.
reply