What's the usage pattern like? Is all of the VRAM used extensively in AI workloads, or one could hope to augment things a bit with system RAM with little performance impact?
You're comparing RAM amounts to other RAM amounts without considering requirements. 24GB is more than (most) current games would ever require, but is considered a uncomfortably-constrictive minimum for most industrial work.
Traditional CPU-bound physics/simulation models have typically wanted all the RAM they could get; the more RAM the more accurate the model. The same is true for AI models.
I can max out 24GB just using spreadsheets and databases, let alone my 3D work or anything computational.
32GB memory only leaves about 24-26GB for the GPU by default which is quite low for a larger model like that. For comparison it runs great on a M2 Max 96GB.
The RTX 4060Ti is the most affordable nVIDIA card with 16GB VRAM from the current generation, making it a good option for AI experimentation. So that might contribute.
24GB is enough for some serious AI work. 48GB would be better, of course. But high end GPUs are still used for other things than gaming, from ML/AI stuff to creative work like video editing, animation renders and more.
2GB VRAM means you can run things comparable to GPT2, a glorified Markov chain. On your CPU you could run much larger models at far from real time speeds.
16 GB is a really nice offering at that price point for AI workloads. I'm keeping my fingers crossed for a higher end Battlemage offering and some real competition for Nvidia.
This is so exciting but the biggest question in my mind is what hardware you'll need to drive all this stuff.
Are they using 3 Nvidia cards in SLI for the demo, or something similarly insane? The 32 GB memory isn't that crazy given how cheap memory is these days.
reply