Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

This depends on the quantization level. Sure, they lose part of the performance, but are still quite useful and cheap to run.


sort by: page size:

They're not the solution to well-sliced data processing, but I can't see how they would hurt the performance.

It can make a difference when using tensor parallelism to run small batch sizes. Not a huge difference, but a noticeable one.

I thought they were useful for embedded use cases, where there isn't much going on so it can take a long time for the OS to collect enough entropy.

They're still a good choice for most perf critical code due to the reduced memory use.

Of course, but if you observe 3x to 10x performance gains across the board, you will have some programs that run in more than 1 second where it may be worthwhile.

Probably a bit. Monomorphization is expensive in terms of code size. On the other hand - why would you care for a couple of 100kb for a use case like transmission?

I mean, if (1) is true but you get faster performance, then it's useful, right?

There just isn't much of a perf penalty. It uses more CPU, yes, but with today's highly-threaded CPUs I can hardly tell there's a cost.

For any data you want to keep, I'd say it's well worth the performance cost. Even for data you don't want to keep, so long as it isn't absolutely performance-critical I find it still worth the cost.


True, but by using binary deltas you can minimize that cost. And failure to upgrade may mean running more inefficient algorithms, wasting more processor cycles, or transferring more data in general. For instance, if you are on a browser that doesn't support Brotli compression then you'll miss out on the dramatic compression improvements provided by that algorithm.

The performance seriously decreases. I haven't measure how much, but I guess it'd only work for very very small scripts.

I'm sure it's important for the demo scene (because the binary size is the only factor), but this is largely irrelevant for modern architectures when performance is considered.

The runtime gain is probably near zero. But there are a handful of advantages.

1. Smaller images. This reduces storage cost, pull time, setup time and can potentially allow more active images to be cached on your nodes.

2. Less installed means less for security scanners to trip on. Even if some software is unused it is good practice, and me be a compliance requirement to update with a fix. If it isn't installed there is no worry.

3. Simplicity. You know that things that aren't installed aren't causing issues. So tiny images can be easier to debug.


Also PoS. Those are the grey area where I don't have as much strong opinions anymore.

As they are less compute-intensive, the benefit is more likely.


I agree entirely - they are great, but in all the Go I've written (which totals about 50kloc so far as a porting project), I haven't actually used them past anything I would have done with zmq in C.

The memory advantages are key to performance when you have a non-trivially sized dataset, in my experience.

In theory, yes, but in practice performance is dominated by network and (less often) algorithms. The cost of boxing/unboxing doesn’t even register except in rare cases, which can be specifically coded for.

At the cost of predictable performance though. With current implementations, the exceptional path will be several magnitudes (!) slower for cheap functions.

It would have a much lower memory and binary size impact. If your program is short-running, it could also be faster.

Yep. The other advantages are that now you can parallelize compute, and you get lower latency.
next

Legal | privacy