TLDR: A guy in Japan who worked on a solution to identify different types of pastries ended up creating a computer vision framework used in many domains. All this without deep learning. The article delves into the challenge that deep-learning brought to his business.
Having until very recently worked in deep learning at Google, I can assure you that if you read and watch enough recent public papers and talks, you will be very, very close to the latest thinking of researchers at these companies.
You're right that it can take some time to do this edification work and develop the understanding for yourself -- the research is broader and more specialized than it appears at first glance -- and it does help to be surrounded by smart people puzzling over the same types of problems, but there's very little secret magic here. It is, however, of benefit to these companies to develop a public image of exclusivity and wizardry in their research; I fell into this trap too, before I saw how the sausage is made.
If you want to make your own fundamental innovations in deep learning, it can be very resource-intensive, both computationally and otherwise. However, it is easy to apply the current state-of-the-art to a broad spectrum of applications in novel ways.
One of the reasons I left is that I think there is a big opportunity in applying these powerful basic principles and approaches to more domains. The research companies are, IMO, focused on businesses that are or have the potential to become very, very large, and that can take advantage of their ability to leverage massive amounts of capital. This leaves many openings for new medium-sized businesses. Of course, as you grow, you can take stabs at progressively larger problems.
Although a title with deep learning in it makes me roll my eyes nowadays, I really should be more open to the ideas behind them. The font pairings at the end of the article look very nice!
Deep learning technologists != people that write articles about machine learning making various professions obsolete. I don’t think anyone that works with neural networks professionally is drinking the kool-aid. Don’t get me wrong, deep learning is an incredible technology, but not in ways that it’s made out to be in the press. Like a lot of other technologies, it’s real value is in solving really boring problems at a scale. But that doesn’t make for good content so we end up with a bunch of articles about self driving cars and nothing about using deep learning to take something like 3d photogrammetry from “passable for artistic purposes” to “industrially valuable.”
>> From the article: “Deep learning, which is fundamentally a technique for recognizing patterns, is at its best when all we need are rough-ready results, where stakes are low and perfect results optional“
> It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence.
No, it's embarrassing how eagerly leaders will compromise quality to chase automation cost savings. The things you list, especially customer service and sports-writing, cannot "be handled by deep learning at its current level of competence." That doesn't stop people from forcing the results down our throats, though.
Deep learning may have become the flavor of the season now, but ironically its pioneers like LeCun and Hinton too faced similar problems when they tried to publish [1].
That is a valid use of deep learning. I think the article is talking about people thinking that because deep learning is very impressive for some things, they should use it for everything - even problems where a much simpler solution works fine, or problems where deep learning doesn't apply.
I work for a consumer product company, and there are often people talking about using 'big data' and 'machine learning'. They're just following the hype; they don't really know what machine learning is and I've only heard two or three potential applications of it mentioned that make any kind of sense.
I love how many paper titles nowadays follow the pattern: "Deep-<topic>: <Actual title of the paper>". And often they aren't doing anything "deeper" than a fully-connected multilayer neural network--a machine learning algorithm competitive with SVMs and been around well over a decade.
Neural Networks weren't really thought of very highly until CNNs started winning image recognition competitions in the early 2010's.
I think most people had the feeling that they were interesting tools to learn how the brain worked, but too slow and opaque to be practical statistical tools. I've been following Hinton for a while (because of Hinton and Shallice 1991), and my understanding is that it was really hard for him to get funding especially when he was just starting out.
The fact that so much of the work from the mid-80's to 2000 came from just a few labs should tell you how hard it was to get funding for that kind of research.
TLDR: A guy in Japan who worked on a solution to identify different types of pastries ended up creating a computer vision framework used in many domains. All this without deep learning. The article delves into the challenge that deep-learning brought to his business.
reply