Always keep in mind that you can’t go faster than what the harware can allow. Sometimes even by what the OS can allow. We can do way better on some specific tasks. But for for that you need to rewrite some algo implementation to better take advantage of all the hardware can allow. ML is a specific task, if you can take advantage of the harware to do better matrix mul. Does not mean you can have the same speed ups overall. Mojo for example
Yea, I was thinking you can't really make a generalization like that.
1. Companies may do more with the same amount of CPU, more data, more calculations, despite per evaluation being faster.
2. Improved performance may make usage within reach of more end users
3. Improving widely used packages could reduce, especially those that are used in heavy calculations. With ML, we often see more improvement (reduced learning time) through algorithm research, so library improvements are only part of the story. Of course advances like Capsule Networks and encrypted learning sit in the queue because they are too expensive to compute today, but have great benefit to the outcomes.
Many 90’s era attempts that failed do work vastly better on modern hardware. There are many tweaks that showed up once people could fine tune using vast amounts of processing power, but I would put ~80% of the advancements from HW and 20% from algorithms.
Having a million or more times the processing power really makes or breaks modern ML.
Chances of you publishing something in ML improve proportionally to the amount of hardware you have access to. Or to put it another way, the less hardware you have, the smarter you have to be to publish something in ML.
this is quite the difference I saw among different ML Engineers during my work.. most can only do python (more or less like a frameworker) and some amazing ones can go all the way down to the CUDA to squeeze out the performance.
Just commenting to agree with this. I've done a lot of ML use case ideation work, and what happens so often is the ML folks latch on to something that ML may be able to do, but that doesn't move the needle at all in terms of doing the work faster or better.
Often, the ML provides an extra layer that slows things down, even if it's working at state of the art AUROC or whatever we're trying for.
Ha, maybe I exaggerate a little! But the ML part seems much more tractable than the hardware side for me! (Mostly because I don't know much about hardware and it seems, well, hard!) :-)
As 'CS type' actually working on ML, I disagree. I'd rather prefer some help from hardware guys, that would allow me to run a neuro network with 100 billion neurons really really fast.
good point and its a big reason why I haven't dived into ML because of the platform I am on is very unfriendly.
Can't do much when you get stuck on step 23 and realize Windows 10 require you to google and look for workarounds.
I'm thinking of doing a completely linux only build with powerful GPU, but this is also where things get tricky because you don't know what you really need, and its a hefty investment when you are not building a machine for fun but for experimentation with AI.
The ML limitations are that all the low level vector and acceleration libraries are written in C or C++. The GPU libraries are the real kicker, there isn't any great way around this except to trampoline over to C, that trampolining takes time which chews into your acceleration. Ironically Python is much faster than Go for ML because of this.
Sure, but it's a continuum, not a binary dichotomy. Just like you can do more with your car if you have degrees in mechanical engineering and fluid dynamics, but a person with nothing but a high-school diploma can upgrade a camshaft.
The point is, you can do a lot of very useful things with ML, without needing the entirety of the theoretical underpinnings. Of course you can't do everything but not everybody needs to be able to do everything.
None. You can be a productive ML engineer without understanding the math. Many elitist engineers here will downvote me, but its true. ML libraries that allow you to quickly get productive have come a long way. BUT, you have to have a solid understanding of WHICH algorithms/tools to use WHEN. There is also a lot of "voodoo" knowledge to gain that isn't well documented or explained (unrelated to maths).
> speedup we need for ML to be truly accessible for the average developer. 50x? 100x?
The data scale (and access) is often more important than the modelling, and that doesn't have a technological fix, or at least not a purely technological fix.
I don't understand your point. When a problem that needs ML materializes, someone uses it to solve said problem. When someone's working on games, firmware, operating systems and compilers, why would they need to come up with creative ways to use ML when they could be doing higher priority stuff (like actually producing a good game?)
Why is ML superior to (say) model checking, numerical analysis and compilers that everyone needs to be doing it?
Its useful for ML in that it requires so much optimization, that being about to drop down in the same language is incredibly handy. For ML software isn't lagging hardware its the other way around.
I don't know why you're being downvoted. At least the last part is totally true; advance in ML is unfortunately just bigger models, more data, and more compute.
But doing ML won't necessarily boost GPU sales because most deep learning work is shifted to the cloud.
reply