Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

It's quite possible that they make heavy use of instancing, which might reduce the memory footprint by a lot. Not every pebble or rock needs an individual geometry.


sort by: page size:

But it's only using as many mappings in total as the memory would be if it had not been compacted.

I wonder if they would run into fragmentation problems with this much of memory, and if so how they are going to deal with that?

But with instancing. So they could have a few dozens of grain types 5K tris each and the total number is from instances, no need to load 5 billion triangles into memory.

Good point; I wonder what kind of memory usage you could squeeze that in to.

Ah, good point, memory consumption might indeed be a factor here. If that's the case I'd expect higher variance in the measurements perhaps - looking for that in the raw data could be interesting.

It's worth noting that this is a comparatively small model (1.6B params from memory).

It'll be interesting what capabilities emerge as they grow that model capacity.


Wouldn't high memory usage suggest it will need to bring more from dusk into memory not less

We’re running 43 clusterio-like connected worlds here. Worlds allocated 8gb ram each. Less if we can get away with it by deleting unused chunks.

Memory footprint?

I think that the memory savings would be very limited, unfortunately.

The current bottleneck for most current hardware is RAM capacity than memory bandwidth and last is FLOPS/TOPS.

The coral has 8 MB of SRAM which uh, won't fit the 2GB+ that nearly any decent LLM require even after being quantized.

LLMs are mostly memory and memory bandwidth limited right now.


Yea, but that doesn't look like what they're describing here. They seem to be describing having the entire dataset in memory.

I'm aware of it, nice project, but 2.6MB + in memory only storage, no thanks.

Your data still has to fit in memory though.

They're using specialized hardware to accelerate their development feedback loop. Without a doubt researchers and hackers will find ways to cut down model sizes and complexity, to run on consumer hardware, soon enough. Just use stable diffusion as an example: 4GB for the whole model. Even if text models are 16GB that'd be great.

It's infinite but not free. Larger context still means more VRAM used and longer compute times.

I dont think of the 4000 tokens as its memory as such. Its more like the size of its thinking workspace

> TSDF memory isn’t an issue since Niessner et al. (2013).

I would strongly disagree. This paper uses TSDF and runs into memory issues. And ATLAS is using TSDF and running into memory issues. So for practical applications, TSDF is still too memory hungry.


Perhaps a better response would be outlining how much memory it actually takes? This way people can decide (i.e. if they care deeply about the memory footprint)
next

Legal | privacy